Installation notes

Support and Help

If you have any problems or suggestions please do get in touch with us at support@ccdc.cam.ac.uk.

An open community forum focused on using the CSD Python API is available at www.ccdc.cam.ac.uk/forum/csd_python_api/ and you are encouraged to visit this forum in the first instance if you require help or inspiration. We are also grateful for any feedback from your experiences with the CSD Python API.

Supported platforms

The CSD Python API is known compatible with, and fully supported on, the following platforms:

  • Windows - Intel compatible, 64-bit versions:
    • Windows 7 and 10
  • Linux - Intel compatible, 64-bit versions:
    • RedHat Enterprise 7 and 8
    • CentOS 7 and 8
  • Mac - Intel compatible:
    • macOS 10.13, 10.14 and 10.15

The CSD Python API may work on additional, unsupported, platforms.

Python version

Python 3.7 is supported, and is provided with the CSDS system. Separate pip and conda packages are distributed for Python 3.7 on supported versions of Windows, Linux and macOS.

For macOS the Python version should be from python.org for our pip package. The pre-installed system Python on macOS is not supported. We recommend the use of our conda packages with Anaconda Python for macOS for macOS.

Python packages are not provided for 32-bit Python.

Installation

Installation options

More than one mechanism exists for installing the CSD Python API.

The CSDS 2021 release includes a self-contained Python environment with the CSD Python API preinstalled with all of its prerequisites. In this case, no additional installation steps are required. We recommend this for most users.

For more advanced users, we also supply both pip and conda packages. These are for the standard Python package managers that are available on most platforms, and may be used for installing the CSD into pre-existing Python environments. The following instructions are specifically for these more complicated options.

Note

In all cases, if you have not yet installed the CSDS 2021 release, then you will need to do this first. However, the toolkit will continue to work in the absence of the CSD if non-data dependent features are used.

Note

It is important to correctly match versions of the ccdc package with the version of Python in use.

Downloads

The downloads required for the CSD Python API are available from the CSDS download page:

  • CSD Python API - Python package
    • pip package or zipped conda channel

Warning

If you previously installed a version of the CSD Python API, you must uninstall this before installing this release. If you used pip, versions later than 1.0 can be uninstalled using:

pip uninstall csd-python-api

The CSD Python API may be installed using either pip or conda. Conda is a part of the Anaconda Python Distribution.

Using conda

  1. Unzip the downloaded conda channel to a location of your choice.

2. Ensure you have installed the CSDS 2021 release in order to use the CSD and any data dependent feature.

  1. Install the packages:
conda install -c <ccdc_conda_channel location> Pillow six lxml numpy matplotlib
conda install -c <ccdc_conda_channel location> csd-python-api

where <ccdc_conda_channel location> is the full absolute path to the unzipped content of the downloaded conda channel.

  1. Finally,on Linux and macOS there are a few more environment variables that must be set to the CSD Python API to communicate with the installed CSD software. CSDHOME must point to the CSD_2021 directory within your CSD Software installation directory, e.g.:

    $ export CSDHOME=/home/my_ccdc_software_dir/CCDC/CSD_2021
    

    On Linux Only, assuming that PYTHONHOME stores the location of the correct Python installation:

    $ export LD_LIBRARY_PATH=$PYTHONHOME/lib:$PYTHONHOME/lib/python3.7/site-packages/ccdc/_lib:$LD_LIBRARY_PATH
    

    On macOS Only:

    $ export DYLD_LIBRARY_PATH=$PYTHONHOME/lib/python3.7/site-packages/ccdc/_lib
    $ export DYLD_FRAMEWORK_PATH=$PYTHONHOME/lib/python3.7/site-packages/ccdc/_lib
    

    It is important that older versions of the CSD Python API are not additionally referenced within the LD_LIBRARY_PATH value (DYLD_LIBRARY_PATH on macOS).

    If you have installed any of the CSD components in non-standard locations it is possible to specify these locations using the environment variables listed below:

    $ # Set the variables required to locate the CSD data and files
    $ export CCDC_TOOLKIT_ASER_DATABASE=/home/my/software/csds_v542/csd/as542be
    $ export CCDC_TOOLKIT_SQLITE_DATABASE=/home/my/elsewhere/as542be_ASER.sqlite
    $ export CCDC_ISOSTAR_DATA_DIRECTORY=/home/my/elsewhere/isostar_files
    $ export CCDC_MOGUL_DATA=/home/my/software/csds_v542/data
    $ export CCDC_MOGUL_INITIALISATION_FILE=$PYTHONHOME/lib/python3.7/site-packages/ccdc/parameter_files/mogul.ini
    $ export CCDC_CROSSMINER_DATABASE=/home/my/elsewhere/crossminer_data/csd_pdb_crossminer.feat
    $ export CCDC_CROSSMINER_FEATURE_DEFINITIONS=/home/my/elsewhere/feature_definitions
    $ export GOLD_DIR=/home/my/elsewhere/Discovery_2021/GOLD
    

Using pip on Windows

  1. If you do not already have a Python 3.7 installation then please download and run the latest Windows installer from www.python.org/downloads. By default this will create:

    C:\Python37
    

    Make sure that your path variable includes your Python directory and your Python Scripts directory. For example by updating the PATH:

    set PATH=%PATH%;C:\Python37;C:\Python37\Scripts
    
  2. If it is not already installed, install pip. pip is included as standard with recent Python versions.

  3. Install the CSD Python API by running:

pip install csd-python-api-3.0.4-win32-py3.7.zip

Warning

If the installation fails to load the required modules with an error similar to Unable to load _UtilitiesLib then it is likely that your computer does not contain the required Microsoft runtime libraries. These may be installed from: the Visual C++ Redistributable Package for Visual Studio 2017

Warning

pip automatically downloads additional packages such as lxml. Some company firewalls block this, so you may need to configure an HTTPS proxy. Ask your systems administrator for more information.

Using pip on Linux

Warning

Using the pip package on macOS is not supported. We recommend the use of our conda packages with Anaconda Python for macOS for macOS.

  1. If your distribution has a package available for Python 3.7, simply install this using the system package manager and skip to step 4; otherwise follow steps 2 and 3 to install Python 3.7 from source.

  2. If you need to install Python from source, first install the libraries required to compile a working Python 3.7. On CentOS 7 and 8, you can install these as follows:

    $ sudo yum install -y gcc make openssl-devel bzip2-devel libffi libffi-devel sqlite-devel xz-devel
    
  3. Next, download the latest Python 3.7 Linux source code package from https://www.python.org/downloads/. To build and install this locally to your user directory:

    $ tar zxvf Python-3.7.*.tgz
    $ cd Python-3.7.*
    $ ./configure --prefix=$HOME/python3.7 --enable-optimizations --enable-shared
    $ make
    $ make install
    

    If you install Python 3.7 to your home directory, make sure to edit your .bashrc file to add the new Python installation to your PATH:

    $ echo export PATH=$PATH:$HOME/python3.7/bin >> ~/.bashrc
    $ echo export LD_LIBRARY_PATH=$HOME/python3.7/lib:$LD_LIBRARY_PATH >> ~/.bashrc
    

    To instead install system-wide to /usr/local - which won’t require changes to your system paths afterwards, but requires root permissions:

    $ tar zxvf Python-3.7.*.tgz
    $ cd Python-3.7.*
    $ ./configure --enable-optimizations --enable-shared
    $ sudo make altinstall
    
  4. To install the CSD Python API into your own Python 3.7 run:

    $ python3.7 -m pip install csd-python-api-3.0.4-linux-64-py3.7.tar.gz
    

    Warning

    pip automatically downloads additional packages such as lxml. Some company firewalls block this, so you may need to configure an HTTPS proxy. Ask your systems administrator for more information.

  5. Finally, there are a few more environment variables that must be set to the CSD Python API to communicate with the installed CSD software. You may again want to add these to your .bashrc to ensure they’re set on every session.

    CSDHOME must point to the CSD_2021 directory within your CSD Software installation directory and CCDC_CROSSMINER_FEATURE_DEFINITIONS to the feature_definitions subdirectory of your CSD CrossMiner installation (if present).

    $ echo export CSDHOME=/home/my_user/CCDC/CSD_2021 >> ~/.bashrc
    $ echo export CCDC_CROSSMINER_FEATURE_DEFINITIONS=/home/my_user/CCDC/CSD_CrossMiner/feature_definitions >> ~/.bashrc
    

    You will also want to set LD_LIBRARY_PATH to include both your Python installation and the CSD Python API _lib directory. On the assumption that PY37 points at the root of your Python 3.7 installation (e.g. /usr/local/ when installing system-wide or the directory in your home directory if you’ve installed locally):

    $ export LD_LIBRARY_PATH=$PY37/lib:$PY37/lib/python3.7/site-packages/ccdc/_lib:$LD_LIBRARY_PATH
    

    It is important that older versions of the CSD Python API are not additionally referenced within the LD_LIBRARY_PATH value.

  6. If you have installed any other CSD components in non-standard locations it is possible to specify these locations using the environment variables listed below:

    $ # Set the variables required to locate the CSD data and files
    $ export CCDC_TOOLKIT_ASER_DATABASE=/home/my/software/csds_v542/csd/as542be
    $ export CCDC_TOOLKIT_SQLITE_DATABASE=/home/my/elsewhere/as542be_ASER.sqlite
    $ export CCDC_ISOSTAR_DATA_DIRECTORY=/home/my/elsewhere/isostar_files
    $ export CCDC_MOGUL_DATA=/home/my/software/csds_v542/data
    $ export CCDC_MOGUL_INITIALISATION_FILE=$PYTHONHOME/lib/python3.7/site-packages/ccdc/parameter_files/mogul.ini
    $ export CCDC_CROSSMINER_DATABASE=/home/my/elsewhere/crossminer_data/csd_pdb_crossminer.feat
    $ export GOLD_DIR=/home/my/elsewhere/Discovery_2021/GOLD
    

Optional Third-party Packages

Several of the example scripts rely on external packages such as matplotlib and jinja2, as do some scripts distributed with the Mercury and Hermes software. These are not needed for use of the CSD Python API itself. To use these example scripts your Python environment will need to include these packages.

To install matplotlib, jinja2, numpy, and biopython, use, e.g.:

pip install matplotlib jinja2 numpy biopython

Testing The CSD Python API

Testing a pip installation

After installing the API from the pip package, its operation may be validated by completing the following steps.

Warning

Some of the tests rely on specific content in the CSD and its updates. It is assumed that the latest CSD and CSD data updates available at the time of the CSD Python API release are installed. If additional updates are applied, or an older release of the CSD is installed, then some test failures are to be expected.

All platforms

  1. Extract the package zip file.

  2. In the top level of the extracted package directory, run:

    python setup.py test
    

    All tests should be marked as PASSED.

    On Linux and macOS if Isostar or Mogul data sources were not defined in installation step 5, above, then some related test failures are expected.

Warning

The CCDCPackageTests.testMatplotlibWorksOnWindows test is known to fail on Windows at this time, and may be ignored. The workaround for the equivalent issue in users’ scripts is documented in the trouble shooting section of this documentation.

Testing the platform

Basic testing of a platform’s suitability for the CSD Python API that catches known problem circumstances may be performed by running a subset of the tests described in the above section. This can be performed with or without installing the CSD Python API. In the latter case, those tests requiring the CSD Python API are skipped.

Within the extracted tests directory, run:

python testCCDCPackageAPI.py