...
| Info | ||
|---|---|---|
| ||
Docker is not supported on TEMS Atos HPCF directly for security reasons. |
You may use Singularity Apptainer if you wish to run containerised workloads. It does not need root privileges to run the containers, and it supports running its own "SIF" container images as well as standard docker containers pulled from any registry such as Docker Hub. Those will get translated automatically automatically into a SIF image before they run. Apptainer is the new name for Singularity.
| Tip | ||
|---|---|---|
| ||
Visit the Singularity Apptainer User Guide for further details |
The basics
First, you you will need to load the singularity module apptainer module so you can start using it:
| No Format |
|---|
$ module load singularityapptainer |
Here's a quick example running a simple command within the official latest Ubuntu Docker image, pulled straight from Docker Hub:
| No Format |
|---|
$ singularityapptainer exec docker://ubuntu:latest cat /etc/os-release INFO: Converting OCI blobs to SIF format INFO: Starting build... Getting image source signatures Copying blob 345e3491a907 done Copying blob 57671312ef6f done Copying blob 5e9250ddb7d0 done Copying config 7c6bc52068 done Writing manifest to image destination Storing signatures 2021/06/07 17:51:35 info unpack layer: sha256:345e3491a907bb7c6f1bdddcf4a94284b8b6ddd77eb7d93f09432b17b20f2bbe 2021/06/07 17:51:36 info unpack layer: sha256:57671312ef6fdbecf340e5fed0fb0863350cd806c92b1fdd7978adbd02afc5c3 2021/06/07 17:51:36 info unpack layer: sha256:5e9250ddb7d0fa6d13302c7c3e6a0aa40390e42424caed1e5289077ee4054709 INFO: Creating SIF file... NAME="Ubuntu" VERSION="20.04.2 LTS (Focal Fossa)" ID=ubuntu ID_LIKE=debian PRETTY_NAME="Ubuntu 20.04.2 LTS" VERSION_ID="20.04" HOME_URL="https://www.ubuntu.com/" SUPPORT_URL="https://help.ubuntu.com/" BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" VERSION_CODENAME=focal UBUNTU_CODENAME=focal |
You can easily get the SIF image file from a docker container with:
| No Format |
|---|
$ singularityapptainer pull docker://ubuntu:latest INFO: Converting OCI blobs to SIF format INFO: Starting build... Getting image source signatures Copying blob 345e3491a907 done Copying blob 57671312ef6f done Copying blob 5e9250ddb7d0 done Copying config 7c6bc52068 done Writing manifest to image destination Storing signatures 2021/06/07 17:51:35 info unpack layer: sha256:345e3491a907bb7c6f1bdddcf4a94284b8b6ddd77eb7d93f09432b17b20f2bbe 2021/06/07 17:51:36 info unpack layer: sha256:57671312ef6fdbecf340e5fed0fb0863350cd806c92b1fdd7978adbd02afc5c3 2021/06/07 17:51:36 info unpack layer: sha256:5e9250ddb7d0fa6d13302c7c3e6a0aa40390e42424caed1e5289077ee4054709 INFO: Creating SIF file... |
Once you have the SIF image, you can run a shell on it :
| No Format |
|---|
$ singularityapptainer shell ./ubuntu_latest.sif Singularity>Apptainer> cat /etc/os-release NAME="Ubuntu" VERSION="20.04.2 LTS (Focal Fossa)" ID=ubuntu ID_LIKE=debian PRETTY_NAME="Ubuntu 20.04.2 LTS" VERSION_ID="20.04" HOME_URL="https://www.ubuntu.com/" SUPPORT_URL="https://help.ubuntu.com/" BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" VERSION_CODENAME=focal UBUNTU_CODENAME=focal Singularity>Apptainer> exit exit |
SIF images are themselves executable, and when run standalone will basically execute what it is defined in their "runscript" section of their singularity their definition file, or just a shell if nothing is defined (such as when pulled from a docker registry):
| No Format |
|---|
$ ./ubuntu_latest.sif Singularity>Apptainer> exit $ ./ubuntu_latest.sif cat /etc/os-release NAME="Ubuntu" VERSION="20.04.2 LTS (Focal Fossa)" ID=ubuntu ID_LIKE=debian PRETTY_NAME="Ubuntu 20.04.2 LTS" VERSION_ID="20.04" HOME_URL="https://www.ubuntu.com/" SUPPORT_URL="https://help.ubuntu.com/" BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" VERSION_CODENAME=focal UBUNTU_CODENAME=focal |
...
By default, the container inherits the environment of the host, and the usual user filesystems are also available by default: $HOME, $PERM, $HPCPERM, $SCRATCH, $SCRATCHDIR and $TMPDIR.
| No Format |
|---|
$ singularityapptainer shell docker://ubuntu:latest INFO: Using cached SIF image Singularity>Apptainer> ls -1d $HOME $PERM $HPCPERM $SCRATCH $SCRATCHDIR $TMPDIR /etcec/ecmwfres4/ssdhpcperm/ssd1user /tmpdirsec/21/usxa.2339821.20210608_133807.854 /home/usxa /lus/pfs1/hpcperm/usxa /lus/pfs1/scratch/usxa /lus/pfs1/scratchdir/usxa/1/at1-11.2339821.20210608_133807.854 /perm/usxa Singularity>res4/scratch/user /ec/res4/scratchdir/user/1/aa6-100.1851137.20220321_133058.868 /etc/ecmwf/ssd/ssd1/tmpdirs/user.1851137.20220321_133058.868 /home/user /perm/user Apptainer> |
You may bind additional directories with the --bind option.
Building your containers
You will not be able to build your singularity images on TEMS from aapptainer images on the Atos HPCF from a definition file, since it requires sudo privileges. However, you can do so on your own computer where you do have root privileges and transfer the final SIF image ready to run. Check Check the Singularity Apptainer User Guide for for more details.
MPI Support
For maximum portability, the recommended approach is the Hybrid Model. That requires installing MPI within the container, and having a compatible MPI or launcher on the host.
...