Using Pip, Apt, and Other Dependencies

For anything more than a simple command, it is likely you will need to specify additional dependencies to get your code to run. There are a number of options for doing so.

Pip

You can add pip packages using the --pip option followed by the name of the package.

$ spell run --pip sklearn "python train.py"

To add multiple packages, you can either add each one using the --pip option or supply a single pip requirements file via the --pip-req option.

$ spell run --pip sklearn --pip imageio "python train.py"

or

$ spell run --pip-req requirements.txt "python train.py"

The requirements file should be a valid pip requirements file. For example:

# requirements.txt
sklearn
imageio

Apt

apt packages are added much in the same way as pip packages, via a command line option: --apt <package>. Spell maintained frameworks are built on the Ubuntu 18.04 Linux distribution, so any package reachable from the default set of package indexes shipped with Ubuntu 18.04 can be specified.

$ spell run --apt libprotobuf-dev --apt protobuf-compiler "python train.py"

Luarocks

To install luarocks packages for torch, install loadcaffe within your run command.

$ spell run --framework torch "luarocks install loadcaffe && python train.py"

Conda

A specific Conda environment for the run can be added via the --conda-file command line option. Both specification files and YAML environment files are accepted. The conda environment name in the run on Spell will be spell.

$ spell run --conda-file ./environment.yml "python train.py".

Docker Hub Public Images

The --from command line option allows you to specify a custom docker image providing the environment for the run. The image must be publicly available on Docker Hub.

Here's an example using OpenPose to detect poses of people in a video.

$ spell run -t K80 --from mjsobrep/openpose:latest \
  "cd /openpose-master; ./build/examples/openpose.bin \
  --video examples/media/video.avi --face --hand --no_display --write_video result.avi"

Python Environments

Python 2 and Python 3 come installed on all Spell environments; however, you can specify the --python2 or --python3 options to control which version of Python the python command is linked to (Python 3 by default if both flags are omitted).

$ spell run --python2 "python train.py"