Python 2 support

Only Python 3 is provided, as Python 2 was officially declared end-of-life by the January 1st, 2020. 

While Python 3 can be found in the system /usr/bin/python3, this version does not come with many of the extra modules you may need. In those cases, you may want to use the python3 module or the conda module (still under development).

The python3 module

This is how Python has been made available on previous ECMWF computing platforms. Newer versions are provided twice a year, and they come with more than 200 extra Python packages. You can get the default version by:

module load python3

virtual environments

If you need to customise your Python environment, you may create a virtual environtment based on the installations provided. This may be useful if you need to use a newer version of a specific python package, but still want to benefit from the rest of the managed Python environment:

module load python3
mkdir -p $PERM/venvs
cd $PERM/venvs
python3 -m venv --system-site-packages myvenv

Then you can activate it when you need it with:

source $PERM/venvs/myenv/bin/activate

The conda module

We also provide a conda module which will allow you to create your own conda environments tailored to your requirements. You can start using conda with:

module load conda

modules and conda incompatibilities

While conda may be seen as a way to set up custom Python environments, it also manages software beyond that, installing other packages and libraries not necessarily related to Python itself.

Because those may conflict with the software made available through modules, loading the conda module effectively disables all the other modules that may be loaded in your environment.

See HPC2020: Conda for more information.

4 Comments

  1. Christopher Roberts

    How do I re-enable modules after unloading conda? In the example below I cannot return to the previous state after loading and unloading conda.

    Load and unload conda test 
    $ module load ecmwf-toolbox
$ module load conda
[...]
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
conda/4.10.1:
 Using conda effectively disables any other loaded modules
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Inactive Modules:
  1) ecmwf-toolbox

$ module unload conda
$ module load ecmwf-toolbox

Lmod has detected the following error:  These module(s) or extension(s) exist but cannot be loaded as requested: "ecmwf-toolbox"
   Try: "module spider ecmwf-toolbox" to see how to load the module(s).

$ module spider ecmwf-toolbox

[...]
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
  For detailed information about a specific "ecmwf-toolbox" package (including how to load the modules) use the module's full name.
  Note that names that have a trailing (E) are extensions provided by other modules.
  For example:

     $ module spider ecmwf-toolbox/2022.03.0.1
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

$ module load ecmwf-toolbox/2022.03.0.1
Lmod has detected the following error:  These module(s) or extension(s) exist but cannot be loaded as requested: "ecmwf-toolbox/2022.03.0.1"
   Try: "module spider ecmwf-toolbox/2022.03.0.1" to see how to load the module(s).

    1. Xavier Abellan

      Best strategy is to reset to default with:

      module reset

      Alternatively, because conda is presented as a prgenv, you can just go back to one of them. For example:

      module load prgenv/gnu

      1. Christopher Roberts

        Thanks Xavi - is it possible to add this info to the message received when loading conda? e.g. change to "Using conda effectively disables any other loaded modules. [Use "module reset" to return to the default environment before loading other modules.]"

        1. Xavier Abellan

          Thanks for the feedback. I have added an extra sentence on the message.