The new MARS client with the MIR interpolation library is used by default at ECMWF since 29 January 2019.

You can use the new MIR-enabled MARS client on any ECMWF computing platform with no extra options or settings.

Please report any issues with this new version of the MARS client to servicedesk@ecmwf.int.

MIR stands for Meteorological Interpolation and Regridding and is a library of routines for interpolation and regridding of meteorological fields. This new piece of software is replacing the veteran EMOSLIB when it comes to perform those operations in MARS.  Beyond this, MIR’s flexible design facilitates scalability improvements and additional features. These include efficiency gains, a high degree of user configurability, and support for a wider range of grids than in the current package. See the related article in the ECMWF newsletter no.152 for a more complete description of this new library.

Table of contents

News

29 January 2019

MARS/MIR becomes the default version at ECMWF. The old MARS/EMOSLIB version can still be accessed specifying the relevant options. See below for details.

1 November 2018

MARS/MIR (1.0.0) is released.  This new version of MARS/MIR is the first version considered ready for operational use. 

24 July 2018 

MARS/MIR (0.10.0) is released.  This new version of MARS/MIR remains a BETA version and is not yet ready for operational use.   It introduces two important changes versus previous versions of MARS/MIR:

  1. Improvements have been made to enhance the performance of spectral to grid transformations when the target is a subarea.
  2. Interpolation of unclassified parameters will fail (this behaviour was reverted in 22 August).

Questions and Answers


The EMOSLIB interpolation package was written in the 1980s and much has changed since then: the model grid resolution has steadily increased, a variety of grid types have been introduced, and many new parameters have been added over the years, often associated with different processing requirements. Both software and hardware technologies, such as programming languages, design paradigms, supporting libraries and hardware architectures, have evolved significantly. These aspects, together with new numerical methods and ECMWF's improved understanding of user requirements, have prompted ECMWF to design the new, extensible and easy-to-maintain MIR package


No, all existing valid MARS requests will also work when using MIR. Some new keywords are introduced for new features such as: 

  • the specification of an intermediate grid to be used when transforming from Spectral Harmonics to lat-lon grids, 
  • custom Land-Sea Mask processing.

You don't need to specify any specific options, as it is the current default.


You can replace your standard mars calls adding the extra option:

mars -e

If using the WebAPI, you can add the keyword:

ppengine=emos

If using Metview greater than 5.4, you need to define the environment variable:

export METVIEW_MARS_INTERP=EMOS

Please let us know if you need to revert to EMOSLIB mode and the reason why by sending an email to servicedesk@ecmwf.int


MIR has undergone a thorough validation process and tests to ensure its quality and correctness. However, since the implementations and strategies used in both packages are different, you may see some differences when using MIR instead of EMOSLIB. How significant those differences will be will depend on each specific case.

Here is an example. The same 2m temperature field on a native O1280 grid has been interpolated to a 0.1/0.1 latitude-longitude grid using both methods. Plots have been produced for both, together with a third one with the differences greater than half a degree Celsius between them.

MARS - LIBEMOSMARS - MIRDifferences greater than 0.5 degree Celsius


At the moment MIR is only made available through the MARS client. In the medium term, a standalone command line tool based on MIR may become available. Other release options may also be considered in the future.


The goal of the first full release of MIR is to replace the old EMOSLIB functionality. As such, MIR implements only linear and nearest-neighbour interpolation methods. It does not provide any support for conservative remapping. Other interpolation methods may be provided in future releases.

This means that MIR may not produce sufficiently accurate results when upscaling from a high-resolution model grid to low resolution grids. For example, using MIR to remap the surface geopotential field from the operational HRES forecast at ~9km to a 1x1 regular lat-lon grid gives a mean bias error of ~16m and a mean absolute error of ~166m when compared to a reference field created by area-averaging the 100 m Shuttle Radar Tomography Data (STRD) grid points over the Alpine Domain. If the the remapping is performed with the the Climate Data Opertors (CDO) remapcon operator.then the mean bias error is ~-2m and the mean absolute error is ~7.5m.  [ECMWF thanks Reto Stöckli of the Federal Office of Meteorology and Climatology MeteoSwiss for providing this information.]

Highlights and main differences

We have compiled a few examples illustrating the differences between EMOSLIB and MIR for a number of representative parameters:

Interpolation methods and distance computation

By default, MIR uses a linear method (based on a triangular mesh) which replaces the EMOSLIB bilinear method (based on 4 points).

MIR also computes distance in 3D whereas EMOSLIB computes distances on the 2D lat-lon grid. Consequently, different nearest points may be selected for the interpolation methods, and the nearest neighbour method should improve as a result.

See Distance metrics for more details.

Parameter classification

In MIR, parameters have been classified into groups,  which have an associated default interpolation method. As a result some parameters have a significantly different default interpolation method compared to EMOSLIB.  Most parameters use the default linear method in MIR, while some specific groups or parameters will use the nearest neighbour method:

  • index parameters: integer values associated to a predefined table
  • ocean wave spectra and associated parameters: intensity as a function of discrete wavelengths
  • vegetation cover ratio parameters

Precipitation

Unlike EMOSLIB, MIR does not use any special processing for precipitation fields.

Subareas

MIR brings a number of improvements in the subarea and cropping operations.

Consistency with fields from dissemination

MIR moves the boundaries inwards onto an underlying global grid with a point at latitude=0, longitude=0. This behaviour is a change in respect to EMOSLIB, and it brings it closer to the product generation for dissemination.

Subareas of reduced Gaussian grids

EMOSLIB only supports subarea operations on regular Gaussian or lat-lon grids. MIR is capable of cropping subareas and frames directly from global reduced Gaussian grids, including the octahedral reduced Gaussian grid.

Support for shifted grids

Shifted Grid

MIR can work with shifted grids, with no point at latitude=0, longitude=0. An example would be: 

AREA=89.5/0.5/-89.5/359.5,
GRID=1.0/1.0

This is done automatically provided the differences between N-S and E-W limits are both divisible by the corresponding increments without remainder. This represents a change in respect to EMOSLIB, which would adjust the area to fit. In the previous example, EMOSLIB would adjust the N/W/S/E boundaries to the more conventional area 90/0/-90/359.

In this example the black points represent the original grid with 1 degree spacing. The red points show the user defined area, which has the same spacing but shifted by half a degree.

MARS with EMOSLIB would expand the area to fit in the original grid, as seen in the yellow rectangle.

MARS with MIR would honour the user defined area and would return the inner grid shown in green.

In cases where the differences between N-S and E-W limits are not divisible by the corresponding increments without remainder, MARS with MIR starts from the user-specified south-west corner and goes up to (but does not exceed) the north-east corner.  For example, a request specifying:

AREA=55.5/0.5/45.0/5.0,
GRID=1.0/1.0

retrieves AREA=55.0/0.5/45.0/4.5 with the northern and eastern boundaries adjusted so that the area returned lies within that specified by the request.

Land-sea mask processing

By default, MIR does not use land-sea mask processing for any parameters. However, users will be able to specify the land-sea mask to use LSM processing for both input and output grids with the new LSM keyword:

LSM=ON

To disable for subsequent requests, once it is enabled:

LSM=OFF

The land-sea mask applied when enabled also changes in respect to EMOSLIB. The new mask in MIR is based on a much finer 1 km resolution field, compared to the 10 minute one used in the past.

MIR does not apply land-sea mask processing when using the nearest neighbour method, which was the case in EMOSLIB.

Grid to grid interpolations

EMOSLIB cannot handle interpolations from a regular latitude-longitude grid to a reduced Gaussian, but MIR does not have that limitation. Interpolations from any global regular or reduced gaussian or latitude-longitude grids to regular or reduced gaussian or regular latitude longitude grids are now supported in MIR.

Spectral to grid point transformations

New keywords to control truncation and intermediate grids

Two new MARS keywords, TRUNCATION and INTGRID, are introduced which supersede the RESOL keyword. RESOL is still supported, but eventually will  be deprecated.

TRUNCATION

Describes how to treat the incoming SH, before the transformation is passed to TRANS library. 

ValueRESOL equivalentDescription
NONEAVDisables any truncation before the transformation
AUTOAUTOTruncation is derived from the intermediate grid. This is the default behaviour

number

numberSpecific truncation to be applied, e.g. TRUNCATION=179 truncates input SH field to T179
OFFOFF reserved MARS value that resets the value inherited, effectively removing the keyword from the request. In this case results in AUTO being applied.

INTGRID

Describes the intermediate grid to which the transform is performed. It may be coincident with the target grid, in which case there isn't a second interpolation and the transform is direct to target grid.

ValueRESOL equivalentDescription
NONE-

Disables the use of an intermediate grid

Use of INTGRID=NONE in combination with the ROTATION keyword is very expensive in resources and is not recommended.

AUTO-An intermediate FULL Gaussian grid is used, derived from the output GRID, by looking at the equivalent resolution. This is the default behaviour

Onumber

OnumberUse the specified Octahedral Gaussian grid as an intermediate grid
FnumberFnumberUse the specified Full (regular) Gaussian grid as an intermediate grid
NnumberNnumberUse the specified Reduced Gaussian grid as an intermediate grid
OFFOFFreserved MARS value that resets the value inherited, effectively removing the keyword from the request. In this case results in AUTO being applied.

Different treatment for RESOL=AV

The concept of "RESOL=AV" ("Archived Value") when going to a lower resolution is different.  With MIR, RESOL=AV specifies that the transformation is made first to the corresponding octahedral reduced Gaussian grid (i.e., T1279 → O1280 or T639 → O640) followed by grid point interpolation to the user-specified grid.

Default truncation values for "RESOL=AUTO" (Automatic Truncation) have also changed. MIR uses a formula to truncate the spectral series to correspond to the equivalent linear Gaussian grid, replacing the fixed table used in EMOSLIB. Transformations to regular latitude-longitude grids then use an intermediate full (regular) Gaussian grid.

EMOSLIB imposes some limitations in the possible combinations of spectral truncation and output grid, only allowing predefined set. These restrictions no longer apply when using MIR

This diagram illustrates an example transformation going from T1279 to a regular lat-lon 1.0/1.0 grid. Depending on the values of RESOL and GRID, MIR will follow different paths. As a reference, the original EMOSLIB behaviour is also shown for each case:

MIR BehaviourEMOSLIB Behaviour
spectral-to-grid


spectral-to-grid-EMOSLIB

PathKeywordsNew keywordsMIR BehaviourEMOSLIB Behaviour
Red

RESOL=AUTO,

GRID=1.0/1.0

TRUNCATION=AUTO,

INTGRID=AUTO,

GRID=1.0/1.0

Default Behaviour.

Transformation from T1279 to an intermediate F90 full (regular) Gaussian grid (with an automatic truncation to T179) followed by a grid-to-grid interpolation from the F90 grid to 1.0/1.0.

Default Behaviour.

Direct transformation from T1279 to 1.0/1.0, with an automatic truncation to T213 according to EMOSLIB's truncation table

Green

RESOL=AV,

GRID=1.0/1.0

TRUNCATION=NONE,

INTGRID=O1280,

GRID=1.0/1.0

Transformation from T1279 to the corresponding intermediate reduced octahedral Gaussian O1280, before going to 1.0/1.0.

Direct transformation from T1279 to 1.0/1.0, with no truncation.
Blue

RESOL=179,

GRID=F90,

and then

GRID=1.0/1.0

TRUNCATION=179,

INTGRID=F90,

GRID=1.0/1.0

Transformation from T1279 to the F90 full (regular) Gaussian grid (with a user-specified truncation to T179) followed by a grid-to-grid interpolation from the F90 grid to 1.0/1.0.Same as in MIR, but a separate MARS request is needed for each of the two steps.

With EMOSLIB spectral fields are by default automatically truncated before interpolation to grid fields to reduce data volumes and spurious aliased values. When automatic truncation is applied MARS issues a warning message like INTFB: Resolution automatically set to ....

The truncation can be controlled using the keywords truncation and intgrid. Users wanting to post-process at the full archived resolution can specify truncation = none in the request.

Legacy EMOSLIB default spectral truncation wave number for given lat/lon grid increment ∆.
Grid increment [deg]Truncation
2.5 ≤ ∆     T63
1.5 ≤ ∆ < 2.5T106
0.6 ≤ ∆ < 1.5T213
0.4 ≤ ∆ < 0.6T319
0.3 ≤ ∆ < 0.4T511
0.15 ≤ ∆ < 0.3 T799
0.09 ≤ ∆ < 0.15T1279
0.0  ≤ ∆ < 0.09T2047

Intermediate Gaussian grids

When doing spectral to grid transformations, MIR may produce smoother fields compared to EMOSLIB. This is due to the fact that MIR uses the intermediate gaussian grid, while EMOSLIB does this transformation directly.


Waves

Due to the differences mentioned above, the nearest neighbour method selects different points due to the difference in the computation of the distance (3D versus 2D). Different treatment of neighbouring missing values produces a smoother interpolation close to coasts.

Also, note that all "Spectra-related" parameters have been classified and are interpolated in the same way as the 2D spectra, using a nearest neighbour in MIR instead of EMOSLIB's bilinear method.

Interpolation from limited-area wave data on a regular lat-lon grid, not working with EMOSLIB,  is now supported in MIR. 

Rotated grids

MIR does not support the 12-point interpolation method, used by EMOSLIB for rotated grids. Therefore, the default interpolation method used is the linear. Rotated spherical harmonics are not supported in MIR or EMOSLIB.

Land-sea mask values

When interpolating the land-sea mask field, MIR returns fractional values greater than or equal to 0 and less than or equal to 1 whereas EMOSLIB returns only 0s and 1s.  Users should regard land-sea mask values greater than or equal to 0.5 as representing land points and those less than 0.5 as representing sea points.

Treatment of values at the poles

MIR is providing different values at the poles to those obtained with EMOSLIB for interpolations to regular latitude-longitude grids which include points at latitudes 90ºN or 90ºS.

Known Issues

See Resolved Issues in MARS/MIR for those issues which have been fixed.

Performance for subareas

When extracting  certain subareas, MIR may be slower compared to EMOSLIB.  Significant enhancements have been implemented in MARS/MIR (0.10.0) which address the performance of spectral transformations to subareas.  Further optimisations for grid point interpolations to subareas may be provided in future releases.



5 Comments

  1. Hi, I just learnt that "--interpolation=grid-box-average" produces a conservative interpolation, but I cannot find that option explained anywhere in the documentation. The "Questions and Answers" also mentions that conservative interpolation is not available yet. Could that be included?


    1. Using INTERPOLATION="--interpolation=grid-box-average" is a way to bypass the MARS client language features/restrictions and passing them directly into the MIR post-processing. There are many other options/interpolation methods available and this mechanism was added so that testing of advanced functionality could be available.

      Support by the MARS client means allowing new methods via the MARS client language, not via this mechanism. If you're interested in having this method, please contact the Production Section which manages the MARS service.

    2. Re-reading your question I suppose you're asking for documentation about this interpolation method? It doesn't exist, but I can create it.

      1. Hi, there is a worked example showing some details of this method when called from Metview here:

        https://metview.readthedocs.io/en/latest/examples/advanced_regrid.html#Conserving-total-precipitation-when-regridding

        As Pedro says, this interpolation method is not currently directly supported by the MARS client, only through this 'back door', but it is supported by Metview - it uses exactly the same code and would give the same results as calling it through the MARS client.

  2. Hello, I am trying to obtain thinned model grid-point data from model spherical harmonics. I've been using trunction=none, intgrid=none, interpolation=nearest_neighbour, grid=O48. I'm just wondering if this is correct, or whether I need to identify the native model grid, and specify this as intgrid? Thanks.