Post-processing of model outputs

  • mslp_${day}.grib for mean sea level pressure,
  • t2_${day}.grib for 2-meter temperature,
  • p_${day}.grib for precipitation,
  • gust_${day}.grib for 10-meter wind gust,
  • t850_${day}.grib for temperature at 850 hPa,
  • q700_${day}.grib for relative humidity at 700 hPa,
  • z500_${day}.grib for geopotential at 500 hPa,
  • u250_${day}.grib for horizontal wind components at 250 hPa,
  • u100_${day}.grib for horizontal wind components at 100 hPa,

where day is the given date in format of yyyymmdd (e.g., 20151203). All the files should contain data for 8 timesteps per day (i.e., in every 3 hours).

Please note the lowercase letters in the filenames.

The mean sea level pressure, the 2-meter temperature, the precipitation and the wind gust are expected in gridpoint representation. So they are prepared from the raw ICMGG* outputs with the next operations:


On this page


Useful links

% grib_copy -w shortName=msl  ICMGG${expID}+00${step} mslp_${day}_${step}.grib  #to get the mean sea level pressure
% grib_copy -w shortName=2t   ICMGG${expID}+00${step} t2_${day}_${step}.grib    #to get the 2-meter temperature
% grib_copy -w shortName=10fg ICMGG${expID}+00${step} gust_${day}_${step}.grib  #to get the 10-meter wind gust

where expID is the 4-digit experiment ID and step is the post-processing step (corresponding to every 3 hours) in 4 digits. For precipitation, both the convective and large-scale precipitation components have to be gathered in the same file:

% grib_copy -w shortName=lsp/cp ICMGG${expID}+00${step} p_${day}_${step}.grib

The pressure level data are required in spectral representation. So they are prepared from the raw ICMSH* outputs with the next operations:

% grib_copy -w shortName=t,level=850 ICMSH${expID}+00${step} t850_${day}_${step}.grib  #to get the temperature at 850 hPa
% grib_copy -w shortName=r,level=700 ICMSH${expID}+00${step} q700_${day}_${step}.grib  #to get the relative humidity at 700 hPa
% grib_copy -w shortName=z
,level=500 ICMSH${expID}+00${step} z500_${day}_${step}.grib  #to get the geopotential at 500 hPa

For wind, both the u and v components have to be collected in the same file:

% grib_copy -w shortName=u/v,level=250 ICMSH${expID}+00${step} u250_${day}_${step}.grib #to get the u and v components at 250 hPa
% grib_copy -w shortName=u/v,level=100 ICMSH${expID}+00${step} u100_${day}_${step}.grib #to get the u and v components at 100 hPa

After these operations, the timesteps belonging to the same days have to be merged into a common file:

% cat ${variable}_${day}_*.grib > ${variable}_${day}.grib

The size of the resulted files varies by the spatial resolution and the representation of the data. For instance, the file size at T255L91 resolution is 10 MB and 8 MB per variables for gridpoint and spectral fields, respectively, whereas these values increases to 35 MB and 26 MB at T639L137, to 233 MB and 179 MB at T1279L137.

Preparation of reference data

  • ${dataset}_mslp_${period}.grib for mean sea level pressure,
  • ${dataset}_t2_${period}.grib for 2-meter temperature,
  • ${dataset}_p_${period}.grib for total precipitation,
  • ${dataset}_gust_${period}.grib for 10-meter wind gust,
  • ${dataset}_t850_${period}.grib for temperature at 850 hPa,
  • ${dataset}_q700_${period}.grib for relative humidity at 700 hPa,
  • ${dataset}_z500_${period}.grib for geopotential at 500 hPa,
  • ${dataset}_u250_${period}.grib for horizontal wind components at 250 hPa,
  • ${dataset}_u100_${period}.grib for horizontal wind components at 100 hPa,

where dataset is a 2-digit identifier of the re-analysis data, ei for ERA-Interim and ea for ERA5; period is the investigated time period in format of yyyymmdd-yyyymmdd (e.g., 20151201-20151206 for Desmond).

Please note again the lowercase letters in the filenames. Furthermore, the re-analysis data should not be split by day, because data for the whole period will be handled together by the Metview macros.

The mean sea level pressure, the 2-meter temperature, the precipitation and the wind gust are expected in gridpoint representation, while the pressure level data are required in spectral representation. Total precipitation and wind gust as parameters representing a time period derive from forecasts, all the other variables are real analyses. Consequently, the daily quantities for precipitation and wind gust are composed of 8 and 24 timesteps from ERA-Interim and ERA5, respectively, the other variables have 4 and 8 timesteps (recall that output frequency of the forecast experiment is 3 hours). Besides the two (large-scale and convective) precipitation components, total precipitation is also available for direct retrieve both in ERA-Interim and ERA5, with GRIB code 228.

The necessary re-analysis data are available from the ECMWF download server in the proper format. For more information please visit http://download.ecmwf.int/test-data/openifs/reference_casestudies.

The re-analysis data can be retrieved directly from the MARS archive, too.

The scr_download_re-analysis shell script is prepared to get the necessary data from MARS. The following steps have to be accomplished:

  1. Using the scr_download_re-analysis script requires Python 2.7 or higher version. Please make sure that the right Python version is installed on your computer.  
  2. ERA-Interim data are available in the ECMWF public datasets. To have access to the ECMWF public datasets, an account is needed to the ECMWF web site: https://apps.ecmwf.int/registration/.
  3. To retrieve data from MARS through ecmwfAPI, an ECMWF key is needed to be downloaded and installed. This page shows that step by step: Access ECMWF Public Datasets.
  4. Having the key, the scr_download_re-analysis shell script has to be run. It is available in the casestudies/mars directory in the OpenIFS cycle 40r1v2 release as well as in the ECMWF download server: http://download.ecmwf.int/test-data/openifs/reference_casestudies/programs/mars. It uses get_ecmars for retrieve which is also located both in the OpenIFS cycle 40r1v2 release (in the bin directory) and in the ECMWF download server.
  5. After submission of the retrieval, it can be checked in the Web-API job list (after login). There is also possibility here to cancel unwanted jobs.

The re-analysis source (i.e., ERA-Interim or ERA5), the surface and pressure level variables to be retrieved, the period of the data and the output directory have to be specified as options for the scr_download_re-analysis script. Calling it with -h option, it provides a detailed help with some examples at the end (but calling it without any option, it also gives a short instruction to its configuration):

% ./scr_download_re-analysis -h

 ----------------------------------------------------------------------
 This script downloads surface and pressure level ERA-Interim or ERA5 re-analysis data for a given time range from MARS.

 Usage:
        -c[class] -s[surface_variables] -p[plevel_variables] -f[firstdate:yyyymmdd] -l[lastdate:yyyymmdd] -o[output_directory]
        -h/-help

 Examples:
        ./scr_download_re-analysis -cei -s t2,p,mslp,gust -p t850,q700,z500,u250,u100 -f20151203 -l20151205 -o../reference
        ./scr_download_re-analysis -cea -s all -p t850,q700,z500,u250,u100 -f20151203 -l20151205 -o/home/rd/digs/metview/paper_OIFS/input/reference
        ./scr_download_re-analysis -ce5 -p t850 -f20151203 -o/home/rd/digs/metview/paper_OIFS/input/reference
  ----------------------------------------------------------------------

Please take into account that the script is able to retrieve only the variables listed above. For further parameters, the program has to be modified manually.

Furthermore, please note the all option which can be used with -s and -p switches. That takes automatically all the surface and/or pressure level variables discussed above (instead of listing them in the command line).

The scr_download_re-analysis shell script is prepared to get the necessary data from MARS. The following steps have to be accomplished:

  1. Using the scr_download_re-analysis script requires Python 2.7 or higher version. Please make sure that the right Python version is installed on your computer.  
  2. Since March 2019, ERA5 data are available through Copernicus Climate Data Store (CDS). To have access to the data, an account is needed to CDS: https://cds.climate.copernicus.eu/user/register.
  3. You will need the CDS API client. It is a python based library, providing support for both Python 2.7.x and Python 3. You can install the CDS API client via the package management system pip, by running on Unix/Linux this command: % pip install cdsapi.
  4. To retrieve data through CDS API, a CDS key is needed to be downloaded and installed. This page shows that: How to use the CDS API.
  5. Before you may download any data from the CDS, you need to have accepted the Terms and Conditions of the chosen dataset. To do this:
    1. Go to the C3S Climate Data Store,
    2. On the top menu bar, click on Datasets,
    3. On the left-hand side menu, expand Product type and select Product type of interest (e.g. Reanalysis for ERA5 datasets),
    4. Follow the dataset title link of interest to the full dataset record,
    5. Accept dataset licence.
     
  6. Having the key, the scr_download_re-analysis shell script has to be run. It is available in the casestudies/mars directory in the OpenIFS cycle 40r1v2 release as well as in the ECMWF download server: http://download.ecmwf.int/test-data/openifs/reference_casestudies/programs/mars. It uses get_ecmars for retrieve which is also located both in the OpenIFS cycle 40r1v2 release (in the bin directory) and in the ECMWF download server.

The re-analysis source (i.e., ERA-Interim or ERA5), the surface and pressure level variables to be retrieved, the period of the data and the output directory have to be specified as options for the scr_download_re-analysis script. Calling it with -h option, it provides a detailed help with some examples at the end (but calling it without any option, it also gives a short instruction to its configuration):

% ./scr_download_re-analysis -h

 ----------------------------------------------------------------------
 This script downloads surface and pressure level ERA-Interim or ERA5 re-analysis data for a given time range from MARS.

 Usage:
        -c[class] -s[surface_variables] -p[plevel_variables] -f[firstdate:yyyymmdd] -l[lastdate:yyyymmdd] -o[output_directory]
        -h/-help

 Examples:
        ./scr_download_re-analysis -cei -s t2,p,mslp,gust -p t850,q700,z500,u250,u100 -f20151203 -l20151205 -o../reference
        ./scr_download_re-analysis -cea -s all -p t850,q700,z500,u250,u100 -f20151203 -l20151205 -o/home/rd/digs/metview/paper_OIFS/input/reference
        ./scr_download_re-analysis -ce5 -p t850 -f20151203 -o/home/rd/digs/metview/paper_OIFS/input/reference
  ----------------------------------------------------------------------

If you have other possibility to retrieve the ERA5 re-analysis data (e.g. via MARS client), please consider to use those one in order to avoid the sometimes long-lasting queues in CDS.

Please take into account that the script is able to retrieve only the variables listed above. For further parameters, the program has to be modified manually.

Furthermore, please note the all option which can be used with -s and -p switches. That takes automatically all the surface and/or pressure level variables discussed above (instead of listing them in the command line).

The retrieved ERA-Interim fields occupy approximately 54 MB, while the ERA5 fields take 700 MB for the Desmond case (i.e., for 1–6 December 2015).

References

Dee, D.P., Uppala, S.M., Simmons, A.J., Berrisford, P., Poli, P., Kobayashi, S., Andrae, U., Balmaseda, M.A., Balsamo, G., Bauer, P., Bechtold, P., Beljaars, A.C., van de Berg, L., Bidlot, J., Bormann, N., Delsol, C., Dragani, R., Fuentes, M., Geer, A. J., Haimberger, L., Healy, S.B., Hersbach, H., Hólm, E.V., Isaksen, L., Kållberg, P., Köhler, M., Matricardi, M., McNally, A.P., Monge‐Sanz, B.M., Morcrette, J., Park, B., Peubey, C., de Rosnay, P., Tavolato, C., Thépaut, J., Vitart, F., 2011: The ERA‐Interim reanalysis: configuration and performance of the data assimilation system. Q.J.R. Meteorol. Soc. 137, 553–597. doi: 10.1002/qj.828

Hersbach, H., Dee, D.P., 2016: ERA5 reanalysis is in production. ECMWF Newsletter 147, p. 7.

Hersbach, H., de Rosnay, P., Bell, B., Schepers, D., Simmons, A., Soci, C., Abdalla, S., Alonso-Balmaseda, A., Balsamo, G., Bechtold, P., Berrisford, P., Bidlot, J-R., de Boisséson, E., Bonavita, M., Browne, P., Buizza, R., Dahlgren, P., Dee, D., Dragani, R., Diamantakis, M., Flemming, J., Forbes, R., Geer, A., Haiden, T., Hólm, E., Haimberger, L., Hogan, R., Horányi, A., Janisková, M., Laloyaux, P., Lopez, P., Muñoz-Sabater, J., Peubey, C., Radu, R., Richardson, D., Thépaut, J-N., Vitart, F., Yang, X., Zsótér, E., Zuo, H., 2018: Operational global reanalysis: progress, future directions and synergies with NWP. ECMWF ERA Report Series 27.


Back to top