Tykky is a set of tools developed at CSC which make software installations on HPC systems easier and more efficient using Apptainer containers. Its main use cases are Python-based conda or virtual environments, and any software or application with a great number of files that when installed on a shared or parallel filesystem may perform very poorly. Tykky will encapsulate those installations in containers, and generate the necessary wrappers so that installed software can be used (almost) as if it was not containerised, improving startup times and reducing the overall load on the filesystems.
To start using Tykky you must first load its module:
module load tykky
Tykky compatibility with modules
Loading tykky will unload all the modules already loaded and will prevent you from loading new ones. This is done to avoid potential conflicts between packages installed in your environments and those from the module system (eccodes, ecflow, compilers, etc.).
To go back to the module system, use the reset command:
module reset
Simply unloading the Conda module won't be enough to go back to the defaults state, many modules won't be visible. It's best to reset the module system.
Containerising a conda environment
You can create a containerised conda environment from an environment file. You can either create it manually or from an existing conda environment with:
conda env export -n <target_env_name> > env.yml
An example env.yml file may look like:
channels: - conda-forge dependencies: - python - ipykernel - cf-units - eccodes - pip - pip: - earthkit
The example would install Python and eccodes from the conda-forge channel, as well as earthkit through pip. You may also pin specific versions if desired.
Once your environment file is ready, you can use Tikky to perform the installation for you. You must decide where to install it, but use of HPCPERM is strongly recommended. Use of HOME, PERM or SCRATCH for such installations is discouraged for performance reasons.
For your convenience, a variable $TYKKY_PATH
is defined by default to use your $HPCPERM/tykky
, that you can use to place your environments:
conda-containerize new --mamba --prefix $TYKKY_PATH/<env_name> env.yml
CPU, Memory and Temporary Storage limitations
If your environment is a complex one, and you are performing the installation from a standard interactive login session, you may hit some limitations in the cpus available, amount of memory or temporary storage.
For example, a typical AI/ML environment may require more than 20 GB of memory and 30 GB of local TMPDIR space. On top of that, creating the image is also a cpu-intensive task that may take some time if run on the default 1 cpu of a login session.
We recommend you to either submit the installation as a batch job specifying higher cpus, memory or storage requirements, or do it interactively with ecinteractive
requesting the necessary resources. Here is an example requesting an ecinteractive
session with 8 cpus, 20 GB of RAM and 30 GB of temporary local disk storage:
ecinteractive -c 8 -m 20 -s 30
When the installation is complete, you will find all the executables in your environment in <install_dir>/bin
.
Using your containerised environment
You may activate your new environment with the convenience tykky activate
function, very similar to how you would do it with conda. If you installed your environment under $TYKKY PATH, then it is as simple as:
tykky activate <env_name>
Of if you installed it somewhere else:
tykky activate <install_dir>
When you are done, you can also deactivate it with:
tykky deactivate
Alternative use
You may alternatively call them directly using the full path, or add it to your PATH
to make them available as if you were activating the environment:
export PATH="<install_dir>/bin:$PATH"
Environment clashes
Your containerised environment may contain a number of executables such as python, which may mask others you may have previously loaded in your environment.
Updating an existing containerised conda environment
If you wish to modify an existing installation, you must create a bash script containing the commands you want to execute to update the installation. They will be executed inside the container, with the environment activated, when you run:
conda-containerize update <install_dir> --post-install update.sh
Here is an example of an update script, which would install metview-batch on top of the existing environment
#!/bin/bash mamba install -y metview-batch
Removing a containerised conda environment
If you no longer need an existing containerised environment, you may simply delete the top level installation directory
rm -rf <install_dir>
Integration with JupyterHub
You can expose your containerised conda environment as an additional kernel in ECMWF's JupyterHub service. To do that, first make sure you have python and ipykernel installed in your environment, and then you can create the kernel with:
<install_dir>/bin/python3 -m ipykernel install --user --name=<my_containerised_env_name>