Introduction

The ECMWF ecCodes software package provides a set of functions/subroutines and command line tools for working with WMO FM-92 GRIB edition 1 and edition 2 messages. The OpenIFS models use GRIB for their input and output files.

The ecCodes software package must be built and installed before compiling any OpenIFS model that needs it.

To download the software please visit the ecCodes website.

If you have a version of ecCodes or the older (no longer supported) ECMWF grib_api library on your system that would most likely work, though check the minimum grib-api version required for each model.  It is recommended that you build and install ecCodes as described below to ensure the correct configuration is used. We also recommend compiling your own to ensure the same compiler and compiler version is used for the model, as otherwise, error messages about conflicting module versions when compiling can arise.

Before you start

Decide where you will put the ecCodes software.

For a single user, ecCodes might be installed in your home directory. For a multi-user environment, say a cluster or High Performance Computing Facility (HPCF), installing ecCodes centrally once only is probably best, then point your build configuration to this location.

In the walk-through example below, a single-user download and install is assumed.

Download and unpack

Create a directory called $HOME/ecmwf in which ecCodes will be downloaded and compiled.

If you haven't already obtained ecCodes, get it from the ecCodes Releases page. We recommend downloading the latest version.

To unpack the software (2.21.0 is given as an example, please use the latest version number):

% mkdir -p ecmwf
% cd ecmwf
% tar zxf eccodes-2.21.0-Source.tar.gz

Configure ecCodes using CMake

ecCodes uses the CMake command for configuration and compilation. CMake is usually installed by default on modern Linux systems but may have to be installed if not.  Note that a cmake version 3.6 and above is required. Some older linux systems may have earlier versions.

CMake compilation is done in a separate build directory to both the source and the install directory. More information on configuring and building ecCodes is available on the ecCodes page.

First create a new directory, "eccodes" in this example, go into it and then run the cmake command to install the build files into that directory. In this example we'll also install the compiled ecCodes into the same directory, but you can change it to by altering the -DCMAKE_INSTALL_PREFIX (e.g. /usr/local/eccodes).

% mkdir eccodes
% cd eccodes
% cmake ../eccodes-2.21.0-Source   \
      -DCMAKE_C_COMPILER=gcc      \
      -DCMAKE_Fortran_COMPILER=gfortran \
      -DCMAKE_INSTALL_PREFIX="$HOME/ecmwf/eccodes"   \
      -DENABLE_NETCDF=ON              \
      -DENABLE_JPG=OFF                \
      -DENABLE_PNG=OFF                \
      -DENABLE_FORTRAN=ON             \
      -DBUILD_SHARED_LIBS=BOTH

Explanation of lines. Note that all options are prefixed by '-D':

Available options

To get additional help on the available options and build configuration run the command:

% cmake -LH eccodes-2.21.0-Source

Change the version number to your installation.

Preferred compiler options

By default, cmake will use the preferred compiler and set appropriate compiler options accordingly (for cmake, this depends on the build type).

This may not be what you need if you have multiple compilers available and want to direct cmake to the most appropriate one to use.

Recommended options given below are conservative to ensure stability and bit-reproducibility.

Changing compiler options is not normally necessary. However, the following examples show the recommended compiler options and how to set them for the cmake commands.

% cmake .... -DCMAKE_C_COMPILER=gcc -DCMAKE_Fortran_COMPILER=gfortran -DCMAKE_C_FLAGS="-g -O2" -DCMAKE_Fortran_FLAGS="-g -O2"

% cmake ... -DCMAKE_C_COMPILER=icc -DCMAKE_Fortran_COMPILER=ifort -DCMAKE_C_FLAGS="-g -O1 -fp-model precise"   \
          -DCMAKE_Fortran_FLAGS="-g -O1 -fp-model precise"

% cmake ... -DCMAKE_C_COMPILER=cc -DCMAKE_Fortran_COMPILER=ftn -DCMAKE_C_FLAGS="-O1 -G2 -hflex_mp=conservative -hadd_paren -hfp1"  \
          -DCMAKE_Fortran_FLAGS="-O1 -G2 -hflex_mp=conservative -hadd_paren -hfp1"

% cmake ... -DCMAKE_C_COMPILER=xlc_r -DCMAKE_Fortran_COMPILER=xlf90_r -DCMAKE_C_FLAGS="-g -O2 -qstrict -qarch=auto -qtune=auto"  \
          -DCMAKE_Fortran_FLAGS="-g -O2 -qstrict -qarch=auto -qtune=auto"

Compiling, checks and installing

After configuration the next steps are to compile ecCodes and install it. To compile ecCodes do:

% make -j2 2>&1 | tee make.out

which will send all the standard and error output from the make command to the file 'make.out' as well as the terminal. This is recommended as the output is lengthy.

The -j2 option will use 2 threads (cores) for a faster compilation.

make VERBOSE=1

% make -j4 2>&1 | tee make.out

Next, to verify ecCodes works correctly run (this will likely download some test data from the ECMWF test-data server):

% ctest -j2 | tee ctest.out

Verify that all the ecCodes tests have passed. If not, go back and check your configuration options. In case of difficulty, contact Software.Support@ecmwf.int.

Finally, to install ecCodes do (the installation directory is specified in the -DCMAKE_INSTALL_PREFIX option (see above):

% make install | tee install.out

At the end of this step, in the directory $HOME/ecmwf/ that we use in this example, you should now have a directory called eccodes which contains the following:

% ls eccodes
bin  include  lib  share
% ls eccodes/lib
libeccodes.a  libeccodes_f90.a  pkgconfig  python2.7

Verify the installation

To verify that the installation was successful, first ensure that the eccodes bin directory is added to your PATH environment variable and then run the codes_info command e.g.

% export PATH=$PATH:$HOME/ecmwf/eccodes/bin
% codes_info

ecCodes Version 2.21.0

Default definition files path is used: /home/myname/ecmwf/eccodes/share/eccodes/definitions
Definition files path can be changed setting ECCODES_DEFINITION_PATH environment variable

Default SAMPLES path is used: /home/myname/ecmwf/eccodes/share/eccodes/samples
SAMPLES path can be changed setting ECCODES_SAMPLES_PATH environment variable

Your output may differ for different ecCodes versions. If the installation was successful, codes_info will report the installation directories.

Possible problems

Cray

Possible issues with the Cray systems (using cce or gnu compilers) are related to building shared libraries and linking with supporting netcdf and python libraries.

1.   If problems occur, first try using the option "-DBUILD_SHARED_LIBS=OFF" to only build statically linked libraries.

2.   If linking problems still occur, try unloading the Cray ATP module which has been known to cause problems.

module unload atp

3.   -llibm link error.

If the following occurs:

/usr/bin/ld: attempted static link of dynamic object `/usr/lib64/libm.so'

set the environment variable CRAYPE_LINK_TYPE=dynamic (or add the compiler option '-dynamic').

4.   Missing python library.

Although it's recommended not to compile with python enabled, if you need this and If this error occurs when compiling _gribapi_swig.so:

/usr/bin/ld: cannot find -lpython

it means the directory path of the python library is not on the linkers list of directories to search.

Make sure the python module is loaded. Then check any PYTHON environment variables set by the module and confirm the location of the libpython (or other version) library.

Set the environment variable LIBRARY_PATH to the correct directory so that the linker can find it:

export LIBRARY_PATH=/home/anaconda/2.2.0-python2/lib64

LIBRARY_PATH is only used at compile/link time. It may also be necessary to add this directory to the LD_LIBRARY_PATH environment variable which is searched for libraries at runtime.

Building on High Performance Computer Systems

Some HPC batch systems have a different hardware architecture for their login (or frontend node) to the batch node, but the frontend compilation system is targeted at the batch nodes. This is known as cross-compilation. If this is the case you may see failures in the 'make check' stage because the checks, although compiled for the backend batch nodes, are being run on the frontend nodes and therefore may not work correctly. If this is the case on your system, we recommend using a batch job to do the 'cmake; make; make check; make install' steps.

Note if you plan on using the eccodes software in your own software that runs on the frontend nodes you may need to compile and install eccodes twice; one for the batch system and again for the frontend system.

If you have any questions installing eccodes in this type of environment, please contact: openifs-support@ecmwf.int for assistance.