HPC2020: ECMWF software and libraries

Some ECMWF tools and libraries have been bundled together in the ecmwf-toolbox package. This now provides, from a single place ecCodes, Magics, Metview and ODC.

Other packages such as ecFlow are still available in their classic standalone modules. If in doubt, you may run module spider <package> to find the right module for you.

In order to use any of those, you just need to do:

module load ecmwf-toolbox

The individual modules for ecCodes, Magics, Metview and ODC are no longer present. You must replace any loads of those modules by ecmwf-toolbox

How to tell what versions of the individual packages are in the toolbox

$ module help ecmwf-toolbox/2020.10.1.2

---------------------------------- Module Specific Help for "ecmwf-toolbox/2020.10.1.2" ----------------------------------
The ECMWF toolbox includes all the software packages and libraries to deal with ECMWF data, such as ecCodes, Magics or Metview.
For more information on the individual packages inside visit https://confluence.ecmwf.int/display/SUP
-- [codes_ui] (1.4.0)
-- [ecmwf-toolbox] (2020.10.1.2) [6a5c9e5]
-- [eccodes] (2.19.1) [929aee7]
-- [bufrdc_tables] (4.1.7) [aff0715]
-- [eckit] (1.13.2) [2732c96]
-- OpenSSL version 1.1.1c -- libs [/usr/lib64/libssl.so /usr/lib64/libcrypto.so] incs [/usr/include]
-- [fckit] (0.9.0) [9cd993a]
-- [odc] (1.1.1) [31b7b8b]
-- [metkit] (1.6.1) [f636381]
-- [odb] (1.1.0) [d4a3156]
-- [odb_tools] (1.1.0) [2f1f7b5]
-- [atlas] (0.22.1) [e55e9c7]
-- [mir] (1.7.1) [98f4b50]
-- [magics] (4.5.1) [bc89653]
-- [mars_client] (6.29.1) [b10a806]
-- [metview] (5.10.1) [e0d2161]
-- system : [at1-118.bullx] [Linux-4.18.0-147.8.1.el8_1.x86_64] [linux.64]

How to tell what version of ecmwf-toolbox provides a specific version of an individual package

$ module spider eccodes/2.19

----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
  eccodes: eccodes/2.19.1 (E)
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    This extension is provided by the following modules. To access the extension you must load one of the following modules. Note that any module names in parentheses show the module location in the software hierarchy.


       ecmwf-toolbox/2020.10.1.2 (prgenv/pgi gcc/8.3.1)
       ecmwf-toolbox/2020.10.1.2 (prgenv/nvidia gcc/8.3.1)
       ecmwf-toolbox/2020.10.1.2 (prgenv/intel gcc/8.3.1)
       ecmwf-toolbox/2020.10.1.2 (prgenv/gnu gcc/8.3.1)
       ecmwf-toolbox/2020.10.1.2 (prgenv/expert gcc/8.3.1)
       ecmwf-toolbox/2020.10.1.2 (prgenv/amd gcc/8.3.1)
       ecmwf-toolbox/2020.10.1.2:gnu:8.3 (prgenv/expert)


Names marked by a trailing (E) are extensions provided by another module.

Using the ecmwf-toolbox from Python

Using the classic python3 installation

Make sure you have the ecmwf-toolbox loaded in your environment along with the python3

module load python3 ecmwf-toolbox

Using a conda environment

You may create your own conda virtual environment and install the desired ECMWF tools there. By default, you would benefit from the predefined ECMWF conda channels containing ECMWF packages built specifically for our HPCF. For example, to create a conda environment with metview:

module load conda
conda create -n myenv metview

See HPC2020: Python support for more details.

Legacy software packages not available

Some of the existing software packages and services installed on our HPC systems, Linux servers and desktop systems in Reading have not been ported on the new platforms in Bologna.

Software/Service	Description	Current Status	To be replaced by	Comment
GRIBEX	GRIB decoder and encoder	Decommissioned	ecCodes
GRIB-API	GRIB decoder and encoder	Deprecated	ecCodes	ecCodes is the default GRIB software loaded on ECMWF's systems. GRIB-API support is being discontinued at the end of 2018
BUFRDC	BUFR decoder and encoder	In use	ecCodes	BUFRDC will be deprecated in Bologna. We recommend users to use ecCodes instead.
SMS	Supervisor Monitor Scheduler	In use	ecflow (version 5 only)	Step by step example of an SMS suite migration to ecFlow.
EMOSLIB	Interpolation software	In use	MIR
metview3	Meteorological workstation application	Deprecated	metview (4 or 5)	metview3 is only installed on ecgate.
Dissemination requirements	Web interface to build the dissemination requirements	In use	New interface
ProdGen	Products generation (for dissemination), based on EMOSLIB	In use	PGEN, based on MIR
ECMWF Metpy	Python OO layer on eccodes	In use	ecmp Metview/Python	ecmp is a reduced version of the current ECMWF's Metpy giving the package a different name; porting to ecmp consists in changing import commands only migrating to Metview/Python would require more work but it is more future-proof (Python3)
ecflowview	Graphical User Interface to ecflow	In use	ecflowUI
Perforce	Version control system	Deprecated	Git

As usually happens when we change HPC system or Linux servers, we will also discontinue some older versions of the remaining software packages installed, e.g. ecCodes, metview, etc

Space shortcuts

Page tree