Cylon has C++ core, Java and Python bindings. You can compile these in three steps.
Here are the prerequisites for compiling Cylon.
- CMake 3.16.5
- OpenMPI 4.0.1 or higher
- Python 3.6 or higher
- C++ 14 or higher
Because Cylon build Apache Arrow with it, we need to specify a Python environment to the build.
If you're using a virtual environment, make sure to set the virtual environment path. Or you can specify /usr as the path if you're installing in the system path.
Create a virtual environment
Here after we assume your Python ENV path is,
We recommend using
Download OpenMPI 4.0.1 from https://download.open-mpi.org/release/open-mpi/v4.0/openmpi-4.0.1.tar.gz
Extract the archive to a folder named
Also create a directory named
buildin some location. We will use this to install OpenMPI
Set the following environment variablesBUILD=<path-to-build-directory>OMPI_401=<path-to-openmpi-4.0.1-directory>PATH=$BUILD/bin:$PATHLD_LIBRARY_PATH=$BUILD/lib:$LD_LIBRARY_PATHexport BUILD OMPI_401 PATH LD_LIBRARY_PATH
The instructions to build OpenMPI depend on the platform. Therefore, we highly recommend looking into the
$OMPI_401/INSTALLfile. Platform specific build files are available in
In general, please specify
--enable-mpi-javaas arguments to
configurescript. If Infiniband is available (highly recommended) specify
--with-verbs=<path-to-verbs-installation>. Usually, the path to verbs installation is
/usr. In summary, the following commands will build OpenMPI for a Linux system.cd $OMPI_401./configure --prefix=$BUILD --enable-mpi-javamake -j 8;make install
Installing with package manager
sudo apt install libopenmpi-dev
We have provided a build script to make the build process easier. It is found in Cylon source root directory.
Build C++, Python Cylon APIs
Cylon provides Python APIs with Cython.
If you're building for the first time, you can use
--all option in build.
If you'have already built cpp and want to compile the your changes to the API,
do the following,
Before running the code in the base path of the cloned repo
run the following command. Or add this to your
- Test Python API