Migration to GRIB edition 2

ECMWF migrated its analysis and forecasting systems to use WMO FM-92 GRIB edition 2 (GRIB-2) encoding for its model level fields on 18 May 2011. This page provides information about this upgrade.

This document will be updated as required and was last changed on 23.05.2011.

For a record of changes made to this page please refer to Document versions.

Migration plan

The encoding of ECMWF model level data in GRIB-2 was implemented with the introduction of IFS cycle 37r2 which includes meteorological as well as technical modifications and improvements.

This new IFS cycle was implemented on Wednesday, 18 May 2011. The first operational run using the new cycle was the 06 UTC analysis and forecast in the Boundary Conditions optional programme (only for members of this programme), followed by the 12 UTC main forecast. The monthly forecast extension to the EPS was run with the new IFS cycle for the first time the following Thursday, 19 May.

The use of GRIB-2 is required later in 2011 for the encoding of an increased number of model vertical levels. The implementation of the increased vertical resolution is planned for late-2011 at the earliest. The adopted migration plan thus separates the GRIB-2 implementation from the vertical resolution change. This will ease the implementation of the increase in vertical resolution as the GRIB-related technical issues will have been resolved by then.

Operationally supported dissemination of GRIB-2 model-level fields has been available for some time already, based on conversion to GRIB-2 of pre-37r2 GRIB-1 data. This option is fully described in

http://www.ecmwf.int/services/dissemination/grib2test.html

Users have been strongly encouraged to make use of this facility in order to test their systems with GRIB-2 data.

Dissemination of test model-level data created directly in GRIB-2 format (without conversion) is now available from the current pre-operational runs - see Test data sets from pre-operational e-suite in dissemination.

What is affected

In brief:

• GRIB-2 will be produced for the deterministic forecast and the EPS, including monthly forecasts, for model level products only.
• GRIB-1 model-level data will no longer be produced and disseminated.
• All other GRIB-1 data should remain unchanged.*
• Comprehensive 37r2 test data is now becoming available.
• There are minor differences between the 37r2 GRIB-2 test data and the pre-37r2 GRIB-2 test data as detailed below.

* Our goal is to have no differences between the GRIB-1 produced by current operations (cycle 36r4) and by the 37r2 E-suite (via grib_api). We have run extensive tests to eliminate any differences. However, should a Member State or other user find any such difference, please contact us so the we can eliminate its cause.

grib_api

The EMOS library GRIBEX routine cannot handle GRIB-2 fields. In order to be able to handle both GRIB-1 and GRIB-2 fields, it is recommended to use grib_api. The latest version of the grib_api library can be downloaded here.

Please let us know if you find any issues with this version of the library.

Interpolation

In order to be able to interpolate GRIB-2 fields, users must upgrade to version 000381 (released on 22 March 2011) or higher of EMOSLIB.

Please note:

• Interpolation of GRIB-2 data has been tested with ECMWF model level fields only.
• In order to interpolate GRIB-2 data, programs need to call intf2 instead of intf
• The intf2 routine also interpolates GRIB-1 data.
• An example program that calls intf2 is available and is also included with the EMOSLIB package in 'examples/interpolation_grib_api'.
• User programs calling the intf2 routine must be linked with version 1.9.9 of the grib_api library - see the compilation instructions provided with the EMOSLIB documentation.

The latest version of EMOSLIB is available here.

Testing of EMOSLIB 000381 with non-model level fields is ongoing. All results so far are unchanged apart from those documented under MARS migration to GRIB API.

Please let us know if you find any issues with EMOSLIB version 000381.

EMOSLIB version 000381 became the default version on ECMWF systems at 09:00 UTC on Tuesday 10 May 2011.

Second-order packing

Encoding of second order packed data in GRIB-1 and GRIB-2 has been implemented in version 1.9.8 of grib_api.

• GRIB-1 second order data encoded with grib_api can be decoded by GRIBEX.
• Encoding of GRIB-2 second order data is not fully compliant with the WMO FM-92 specifications and therefore a local template has been designed for it. For further details, see GRIB API keys for GRIB edition 2.
• Decoding of second order packed data produced by GRIBEX has been fully implemented in previous grib_api versions.

Latitude and longitude values for reduced Gaussian grids in GRIB-2

Latitude and longitude values of the first and last point are encoded in millidegrees in GRIB-1 and in micro-degrees in GRIB-2. As those values are encoded with the highest possible accuracy in each GRIB edition users should expect to get different values in GRIB-1 and GRIB-2 fields.

The latitudes and longitudes for each grid point can be obtained from grib_api with high accuracy using the keys latitudes, longitudes, distinctLatitudes, distinctLongitudes or using the grib_get_data Fortran functions or the grib_iterator C functions.

As grib_api is computing latitudes and longitudes from the order N of the Gaussian grid the values are identical in both editions and they are very accurate. Definitions of the various Gaussian grids in use at ECMWF can be found here.

Coding of negative longitude values in GRIB-2

According to the WMO FM-92 GRIB edition 2, only positive values of longitudeOfFirstGridPoint, longitudeOfLastGridPoint, longitudeOfSouthernPole are allowed. Negative values of longitudeOfFirstGridPointInDegrees, longitudeOfLastGridPointInDegrees, longitudeOfSouthernPoleInDegrees are converted by grib_api to positive values by adding 360 to the value before encoding.

GRIB-1 to GRIB-2 conversion

Data can be converted from GRIB-1 to GRIB-2 using grib_api. ECMWF is also providing a conversion tool for this purpose. The tool is currently limited to the conversion of model level data, but will be extended to convert more level types and parameters, as their GRIB-2 encoding is agreed by WMO.

The tool to use in order to convert from GRIB-1 to GRIB-2 for model levels only is called 'grib1to2' and is available on all of the computing systems at ECMWF:

usage: grib1to2 in.grib1 out.grib2

At this stage the tool is working only with ECMWF model level data (mars.levtype=ml, typeOfLevel='hybrid'). If data with type of level different from "hybrid" are processed the conversion tool fails printing an error message.

GRIB-2 to GRIB-1 conversion

A general tool for conversion back from GRIB-2 to GRIB-1 cannot be provided, as this might lead to a loss of information. However, grib_api tools can be used to create scripts to convert subsets of data and ECMWF is available to provide limited advice for the development of such conversion scripts, if requested.

Retrieving GRIB-2 data from MARS

The MARS application has been migrated to use grib_api as its data decoder instead of GRIBEX. To retrieve data in GRIB-2 format the use of this new version is essential.

For a list of known differences between the GRIBEX and the grib_api version of MARS please refer to MARS migration to grib_api.

The grib_api version of MARS became the default version on all ECMWF systems at 09:00 UTC on Tuesday 10 May 2011.

Support for GRIB-2 fields in Metview 3 at ECMWF

A version of Metview 3 with full GRIB 2 support for interpolations has been made available on ecgate.

This version of Metview 3 became the user version of Metview 3 (command 'metview') at 09:00 UTC on Monday, 16 May 2011.

Locally installed versions of Metview

Users of Metview 3 wishing to acquire full GRIB 2 support for interpolations should upgrade to version 3.12.1 or higher. This version contains the latest EMOSLIB library and needs to be linked with grib_api 1.9.9.

Users of the newly developed Metview 4 should use version 4.0.3 or higher to get full GRIB 2 support. This version is not yet used operationally at ECMWF.

All versions of Metview are available on request from Software Services at ECMWF.

Locally installed versions of the MARS client

Member States using a locally installed version the MARS client to retrieve model level fields will need to upgrade their software. An updated version of the MARS client with support for model level fields in GRIB-2 is available upon request from Software Services at ECMWF..

MAGICS 6 and GRIB-2 fields

MAGICS 6 is NOT able to handle GRIB 2 fields since it depends on GRIBEX to perform the GRIB handling. Users are advised to migrate to Magics++ or, as is done with Metview 3, convert the fields from GRIB-2 to GRIB-1 before processing with MAGICS 6.

PrepIFS experiments

PrepIFS experiments for IFS cycle 37r2 and above will produce data in GRIB-2 for model level parameters and GRIB-1 for the rest.

ECMWF applications to be discontinued

Two obsolete applications will be discontinued. They are:

• The trajectory module in Metview 3 and the corresponding trajectory data base will not be migrated to GRIB-2. This service has been hosted by ECMWF and the externally provided maintenance of the software ceased several years ago. The trajectory data base will thus be discontinued, which means that any user application that still accesses this data will stop working when IFS cycle 37r2 is implemented.
• The 'multi-analysis suite' producing forecasts from other global NWP centres' analyses will be discontinued. Such data was stored in MARS as stream = AMAP / MAED / MAWV

Minor encoding differences in the pre-operational e-suite GRIB-2 fields

Due to a different numerical representation between GRIB-1 and GRIB-2, the "a" and "b" coefficients (the "pv" grib_api key ) describing the vertical levels are coded with higher accuracy in GRIB-2 and are therefore closer to the documented values. This is not the case when converting from GRIB-1 to GRIB-2 as those coefficients are already in a low accuracy representation in the GRIB-1 message and the higher accuracy values cannot be recovered from it. As the pre-37r2 dissemination test data are produced by conversion from GRIB-1 to GRIB-2 the a and b coefficients in them are coded with low accuracy. These small differences have no significant meteorological impact.

Other minor differences have been identified in the encoding of the GRIB-2 fields between the test data sets that have been provided to-date in dissemination or created by using the grib1to2 tool and those from the IFS cycle 37r2 pre-operational e-suite. These differences are summarized in the following tables.

If any other encoding differences are found, they will be reported in this page.

Migration mailing lists

ECMWF maintains two mailing lists to support you with your migration to GRIB-2:

• Please send email to grib2-migration@lists.ecmwf.int to report GRIB-2 migration problems to us.
• Please subscribe to grib2-migration-announce@lists.ecmwf.int to receive information about changes and bug fixes.
  
  To subscribe, either send an email to sympa@lists.ecmwf.int with "subscribe grib2-migration-announce" as the subject or go to http://www.ecmwf.int/publications/maillist/d/subscribe/grib2-migration-announce and enter your email address.

Document versions