See HPC2020: The Lmod Module system for a complete picture
A number of software, libraries, compilers and utilities are made available through the HPC2020: The Lmod Module system.
If you want to use a specific software package, please check if it is already provided in modules. If a package or utility is not provided, you may install it yourself in your account or alternatively report as a "Problem on computing" through the ECMWF Support Portal.
Example: finding relevant modules for NetCDF libraries:
module spider netcdf
The traditional module avail will only show the modules available for the loaded programming environment.
And then, load the desired module with
module load netcdf4
You may need to load a prgenv first if the desired package is compiler sensitive (i.e. providing libraries to users).
If a package or utility is not provided, you may install it yourself in your account or alternatively raise an issue through the ECMWF Support Portal
HPC2020: The Lmod Module system
For the complete list of options and commands, see the Lmod modules command reference list
Lmod is a Lua-based implementation of the module system, and generally claimed to be backwards compatible with the original. On the newer ECMWF platforms it is replacing the traditional TCL modules.
It is used to configure and manage your session or job environment with all those tools and libraries required for your workload.
HPC2020: Compilers
Several compiler families are installed on the Atos HPCF. In particular, you will find several versions of the GNU, Intel and AMD AOCC compilers.
All the support libraries will be providing an installation for each one of those flavours.
Managing those flavours is easy with modules and the prgenv module.
HPC2020: Building your programs
When you log in, the GNU prgenv with GCC would be loaded by default, but can be easily changed. For example, to use the Intel Programming Environment or toolchain:
module load prgenv/intel
See HPC2020: Compilers and The prgenv module for more details on how to customise your build environment and toolchains.
HPC2020: MPI
Several MPI implementations are available on the Atos HPCF: OpenMPI (provided by Atos), Intel MPI and Mellanox HPC-X (OpenMPI based).
They are not compatible amongst them, so you should only use one to build your entire software stack. Support libraries are provided for the different flavours to guarantee maximum compatibility.
HPC2020: ECMWF software and libraries
Some ECMWF tools and libraries have been bundled together in the ecmwf-toolbox package. This now provides, from a single place ecCodes, Magics, Metview and ODC.
Other packages such as ecFlow are still available in their classic standalone modules. If in doubt, you may run module spider <package> to find the right module for you.
HPC2020: Python support
Python 2 support
Only Python 3 is provided, as Python 2 was officially declared end-of-life by the January 1st, 2020.
While Python 3 can be found in the system /usr/bin/python3, this version does not come with many of the extra modules you may need. In those cases, you may want to use the python3 module or the conda module (still under development).
HPC2020: Debugging
There are a few options on the Atos HPCF when it comes to debugging your malfunctioning program:
HPC2020: ARM DDT
Arm DDT is a powerful, easy-to-use graphical debugger. It provides a complete solution for finding and fixing problems on a single thread or across hundreds of thousands of threads. You can use Arm DDT to debug OpenMP, parallel (MPI) software, heterogeneous software such as that written to use GPUs, hybrid codes mixing paradigms such as MPI + OpenMP, or MPI + CUDA, and multi-process software of any form, including client-server applications. Please refer to the Official DDT documentation to learn how to use it.
HPC2020: Container support
Docker support
Docker is not supported on Atos HPCF directly for security reasons.
You may use Apptainer if you wish to run containerised workloads. It does not need root privileges to run the containers, and it supports running its own "SIF" container images as well as standard docker containers pulled from any registry such as Docker Hub. Those will get translated automatically into a SIF image before they run. Apptainer is the new name for Singularity.