Cylon has C++ core, Java and Python bindings. You can compile these in three steps.
Here are the prerequisites for compiling Cylon.
- CMake 3.16.5
- OpenMPI 4.0.1 or higher
- Python 3.6 or higher
- C++ 14 or higher
Because Cylon build Apache Arrow with it, we need to specify a Python environment to the build.
If you're using a virtual environment, make sure to set the virtual environment path. Or you can specify /usr as the path if you're installing in the system path.
Create a virtual environment
Here after we assume your Python ENV path is,
We recommend using
Download OpenMPI 4.0.1 from https://download.open-mpi.org/release/open-mpi/v4.0/openmpi-4.0.1.tar.gz
Extract the archive to a folder named
Also create a directory named
buildin some location. We will use this to install OpenMPI
Set the following environment variablesBUILD=<path-to-build-directory>OMPI_401=<path-to-openmpi-4.0.1-directory>PATH=$BUILD/bin:$PATHLD_LIBRARY_PATH=$BUILD/lib:$LD_LIBRARY_PATHexport BUILD OMPI_401 PATH LD_LIBRARY_PATH
The instructions to build OpenMPI depend on the platform. Therefore, we highly recommend looking into the
$OMPI_401/INSTALLfile. Platform specific build files are available in
In general, please specify
--enable-mpi-javaas arguments to
configurescript. If Infiniband is available (highly recommended) specify
--with-verbs=<path-to-verbs-installation>. Usually, the path to verbs installation is
/usr. In summary, the following commands will build OpenMPI for a Linux system.cd $OMPI_401./configure --prefix=$BUILD --enable-mpi-javamake -j 8;make install
Installing with package manager
sudo apt install libopenmpi-dev
We have provided a build script to make the build process easier. It is found in Cylon source root directory.
Build C++, Python Cylon APIs
Cylon provides Python APIs with Cython.
If you're building for the first time, you can use
--all option in build.
If you'have already built cpp and want to compile the your changes to the API,
do the following,
Note: You only need to do
--python just once after the initial C++ build. If you develop the
Cython or Python APIs, use
--cython flag instead.
Before running the code in the base path of the cloned repo
run the following command. Or add this to your
- Test Python API