Implementation of IFS Cycle 47r3 on Atos

Differences in IFS forecasts produced in parallel on CRAY and ATOS systems

Since the ATOS esuite was first introduced the two IFS systems (CRAY and ATOS) have been running independently. Whilst both are fed with the same input data, as the days and weeks pass some minor differences will emerge in the analyses, and therefore in the forecasts (partly linked to the different HPC platforms). Indeed the differences within forecast pairs grow with lead time. These differences are not however indicative of a problem on the ATOS (or the CRAY) but are symptomatic of innate analysis uncertainties. They should be viewed as salutary - highlighting that one cannot rely on deterministic run details, even at short leads. We have also confirmed that statistically the CRAY and ATOS runs are equivalent, with neither better nor worse than the other.

Implementation plan

To switch the operational forecast to the Atos machine we will use a process similar to the implementation of new model cycles. During the Release Candidate Phase we will run a frozen test system in parallel and close to the operational schedule. A Release Candidate Phase of at least 3 months will give forecast users the opportunity to not only test the model output itself but use it to test all other components of the new infrastructure in Bologna. While the model output data on the Atos machine will be different from that currently produced operationally on the Cray, it is expected to be meteorologically equivalent.

Implementation: 06 UTC run on 18 Oct 2022

Indicative timeline 

Technical content

Users should not see any technical differences between the Atos IFS Cycle 47r3 data and products and current operational ones produced on our Cray systems in Reading. Users can continue using their existing software packages to decode the Atos IFS Cycle 47r3 data.  See 47r3 software requirements for more details.

Atos IFS Cycle 47r3 test data and products

Test data in MARS

IFS Cycle 47r3 Release Candidate test data is available from MARS with experiment version 0077 (MARS keyword EXPVER=0077): 

• HRES (class=od, stream=oper, expver=77)
• HRES-WAM (class=od, stream=wave, expver=77)
• ENS (class=od, stream=enfo, expver=77)
• ENS-WAM (class=od, stream=waef, expver=77)
  
  
• HRES at 6/18 (class=od, stream=scda, expver=77)
• HRES-WAM at 6/18 (class=od, stream=scwv, expver=77)
  
  
• SEAS (class=od, stream=mmsf, expver=9001, from August) 

Only users registered with access to MARS will be able to access these test data sets. The data should not be used for operational forecasting.  Please report any problems you find with this data via the ECMWF Support Portal.

The Release Candidate test data in ECPDS, available from 19 Jul 2022, will be generated daily, shortly behind operational high resolution and ensemble runs and based on the operational dissemination requirements. The availability of the test data does not follow any strict schedule.

For longer time series available beta test data with EXPVER=9077 from 15 Feb 2022 can be used.  For access to CAMS data please see Accessing CAMS 47r3 technical test data

Test data in dissemination

Cycle 47r3 test data from the Release Candidate Phase will be available through the test dissemination system, starting from the 00Z run on 19 Jul 2022. Users with access to ECPDS can login to the test system in Bologna at https://xdiss-monitor.ecmwf.int/do/start and trigger the transmission of test products in the usual manner (for ECPDS access to CAMS please see Accessing CAMS 47r3 technical test data). To receive the test data from Bologna, we will remind users to have their firewall open to the relevant ECPDS Data Movers (cf. https://confluence.ecmwf.int/x/FyFEDg):

• Internet:

  • 136.156.192.0/26
  • 136.156.193.0/26

  RMDCN:

  • 136.156.196.0/26
  • 136.156.197.0/26

The Atos IFS Cycle 47r3 test products are available as version number 77 (file names ending with '77' with the ecCodes key expver=77 in the GRIB messages) and are intended to be generated shortly behind real-time. The test products will be based on the operational dissemination requirements and updated in near real time when changes to the requirements are made. The test products will be available for HRES, HRES-WAM, ENS, ENS-WAM, ENS extended and SEAS. Test data from the Boundary Condition program, HRES and ENS runs at 06:00 and 18:00 UTC, is also available.

Access to the operational PREd will be closed at 9 UTC on 16 October. Any changes to the operational requirements can be implemented after 10 UTC on 18 October.

If you don't have access to the ECPDS system or PREd or should you require any assistance with Atos IFS Cycle 47r3 test dissemination products, please contact us via the Support Portal.

Graphical display of test data in ecCharts

For this technical change it is not planned to make IFS cycle 47r3 Release Candidate test data available in ecCharts. 

Web charts based on test data

For this technical change it is not planned to make available OpenCharts based on the Atos IFS cycle 47r3 Release Candidate test data.

WMO Essential and Additional test data

IFS cycle 47r3 WMO Essential test data starting from the 00Z run on 18 Jun 2022is available at https://xdiss.ecmwf.int/ecpds/home/wmo/.

Time-critical applications

Option 1 - simple time-critical jobs

Member State users of the  "Simple time-critical jobs" framework can test that their scripts/jobs will work with the IFS Cycle 47r3 test data on the Atos by using the limited ECaccess 'events' set up for this purpose. 

For these events, MSJ_EXPVER environment variable is set to 0077 and can be used to specify the IFS Cycle 47r3 test data in any MARS retrievals. These events are intended for testing technical aspects only and should not be used for operational time-critical activities. Please note that ECaccess events for the operational production (expver=1) are also available, both in Reading and Bologna. 

For data transfers using ectrans remote sites are advised to upgrade their ECaccess gateways with the latest gateway package to be able to connect to the ECMWF gateway based in Bologna, boaccess.ecmwf.int, via the Internet. 

Options 2 and 3

Option 2 or 3 time-critical applications can be tested with the Atos IFS Cycle 47r3 test data retrieved from MARS or received in Dissemination.