Docker support

Docker is not supported on Atos HPCF directly for security reasons.

You may use Apptainer if you wish to run containerised workloads. It does not need root privileges to run the containers, and it supports running its own "SIF" container images as well as standard docker containers pulled from any registry such as Docker Hub. Those will get translated automatically into a SIF image before they run. Apptainer is the new name for Singularity.

Official reference documentation

Visit the Apptainer User Guide for further details

The basics

First, you will need to load the apptainer module so you can start using it:

$ module load apptainer

Here's a quick example running a simple command within the official latest Ubuntu Docker image, pulled straight from Docker Hub:

$ apptainer exec docker://ubuntu:latest cat /etc/os-release
INFO:    Converting OCI blobs to SIF format
INFO:    Starting build...
Getting image source signatures
Copying blob 345e3491a907 done  
Copying blob 57671312ef6f done  
Copying blob 5e9250ddb7d0 done  
Copying config 7c6bc52068 done  
Writing manifest to image destination
Storing signatures
2021/06/07 17:51:35  info unpack layer: sha256:345e3491a907bb7c6f1bdddcf4a94284b8b6ddd77eb7d93f09432b17b20f2bbe
2021/06/07 17:51:36  info unpack layer: sha256:57671312ef6fdbecf340e5fed0fb0863350cd806c92b1fdd7978adbd02afc5c3
2021/06/07 17:51:36  info unpack layer: sha256:5e9250ddb7d0fa6d13302c7c3e6a0aa40390e42424caed1e5289077ee4054709
INFO:    Creating SIF file...
NAME="Ubuntu"
VERSION="20.04.2 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04.2 LTS"
VERSION_ID="20.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal

You can easily get the SIF image file from a docker container with:

$ apptainer pull docker://ubuntu:latest
INFO:    Converting OCI blobs to SIF format
INFO:    Starting build...
Getting image source signatures
Copying blob 345e3491a907 done  
Copying blob 57671312ef6f done  
Copying blob 5e9250ddb7d0 done  
Copying config 7c6bc52068 done  
Writing manifest to image destination
Storing signatures
2021/06/07 17:51:35  info unpack layer: sha256:345e3491a907bb7c6f1bdddcf4a94284b8b6ddd77eb7d93f09432b17b20f2bbe
2021/06/07 17:51:36  info unpack layer: sha256:57671312ef6fdbecf340e5fed0fb0863350cd806c92b1fdd7978adbd02afc5c3
2021/06/07 17:51:36  info unpack layer: sha256:5e9250ddb7d0fa6d13302c7c3e6a0aa40390e42424caed1e5289077ee4054709
INFO:    Creating SIF file...

Once you have the SIF image, you can run a shell on it :

$ apptainer shell ./ubuntu_latest.sif 
Apptainer> cat /etc/os-release 
NAME="Ubuntu"
VERSION="20.04.2 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04.2 LTS"
VERSION_ID="20.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal
Apptainer> exit
exit

SIF images are themselves executable, and when run standalone will basically execute what it is defined in their "runscript" section of their definition file, or just a shell if nothing is defined (such as when pulled from a docker registry):

$ ./ubuntu_latest.sif 
Apptainer> exit
$ ./ubuntu_latest.sif cat /etc/os-release 
NAME="Ubuntu"
VERSION="20.04.2 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04.2 LTS"
VERSION_ID="20.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal

Environment and filesystems in the container

By default, the container inherits the environment of the host, and the usual user filesystems are also available by default: $HOME, $PERM, $HPCPERM, $SCRATCH, $SCRATCHDIR  and $TMPDIR. 

$ apptainer shell docker://ubuntu:latest
INFO:    Using cached SIF image
Apptainer> ls -1d $HOME $PERM $HPCPERM $SCRATCH $SCRATCHDIR $TMPDIR /ec/res4/hpcperm/user
/ec/res4/scratch/user
/ec/res4/scratchdir/user/1/aa6-100.1851137.20220321_133058.868
/etc/ecmwf/ssd/ssd1/tmpdirs/user.1851137.20220321_133058.868
/home/user
/perm/user
Apptainer>

You may bind additional directories with the --bind option.

Building your containers

You will not be able to build your apptainer images on the Atos HPCF from a definition file, since it requires sudo privileges. However, you can do so on your own computer where you do have root privileges and transfer the final SIF image ready to run. Check the Apptainer User Guide for more details.

MPI Support

For maximum portability, the recommended approach is the Hybrid Model. That requires installing MPI within the container, and having a compatible MPI or launcher on the host.

Performance note

Installing stock MPI implementations from official repositories will usually work, but they may not be able to fully exploit the Infiniband Fabric for inter-node communication. If you intend to run MPI workloads across multiple nodes, make sure you build the appropriate support for the underlying fabric.