The continuing sequence of analyses

Each ECMWF forecast, atmospheric, wave or oceanic, does not stand alone.  It builds upon an analysis incorporating the latest data together with output from a previous IFS model run.

It is necessary to balance:

  • incorporating as many observations as possible to get the best possible analysis.
  • forecasts must be produced sufficiently quickly so that they can be used effectively and not be too late to be of use. 

Observations are made at different times and arrive after varying delays.  Waiting for later data benefits the quality of the analysis and its subsequent forecast, but adversely affects product timeliness.  To overcome this problem, ECMWF uses an early delivery system.  This allows the 00UTC and 12UTC operational analyses to be produced significantly earlier without compromising the operational quality of the forecast products very much.

Fig2.6-1: Simplified overview of Observation - Analysis - Forecast sequence in IFS.  Observations within the appropriate window are assimilated using 4D-var and the subsequent forecast is used to supply initial conditions and background (first guess) fields for the next 4D-var assimilation.  The sequence is repeated in an ongoing cycle at 6 hourly intervals - each forecast providing the background field for the following IFS models into which the latest observations are incorporated using 4D-Var. 

ECMWF analyses the data available within two periods and for different reasons.  The sequence of model data assimilation using 4D-Var that is followed by IFS forecast system is explained in more detail below.    

The Long Window Data Analysis (LWDA):

The LWDA output fields constitute a stand-alone system for accurately analysing the evolving state of the atmosphere, but are generally not directly used by operational forecasters.  Forecasts are only run to relatively short lead times for use as background fields in the 4D-Var analysis process for subsequent analysis processes 6hr and 12hr later.  The LWDA analysis process allows the maximum use of available data but means the analyses are delayed significantly from the nominal data time.  The LWDA process involves continuous data assimilation and can be referred to as Co-LWDA.  (The Long Window Data Analysis was previously referred to as the "Delayed Cut-Off Analysis" but the term is now obsolete).

The analysis starts 14 hours after the nominal model data time.  This late cut-off is sufficiently late to allow inclusion of virtually all available (including delayed) observations relevant to the IFS model run.  Thus:

    • the 00UTC analysis starts at 14UTC and uses all observations having a time within the 21-09UTC assimilation period and arriving before 14UTC.

    • the 12UTC analysis starts at 02UTC and uses all observations having a time within the 09-21UTC assimilation period and arriving before 02UTC.

The process is repeated at 12 hour intervals.

For 00UTC LWDA:  

  • The 12UTC 4D-Var LWDA uses observations with data times between 09UTC and 21UTC to produce the best available 12UTC analysis.
  • Also, the LWDA uses the observations with data times between 15UTC and 21UTC to produce an 18UTC analysis.  This is the basis for a short 18UTC forecast to T+15hr (to 09UTC).
  • Forecast fields T+03hr to T+09hr from the 18UTC forecast (21UTC to 03UTC) are used as first guess and background fields for the 00UTC SCDA.
  • The first guess for the 00UTC 4D-Var LWDA analyses is provided by the analysis from the 00UTC SCDA which has already used all available information with data times between 21UTC and 03UTC (T+03hr and T+09hr from the 18UTC forecast), including incorporation of late arrival data by Continuous Data Assimilation (CoDA).
  • Background fields during the rest of the 4D-Var analysis is provided by T+12hr and T+15hr output from the 18UTC forecast (06UTC and 09UTC).

For 12UTC LWDA:

  • The 00UTC 4D-Var LWDA uses observations with data times between 21UTC and 09UTC to produce the best available 00UTC analysis.
  • Also, the LWDA uses the observations with data times between 03UTC and 09UTC to produce an 06UTC analysis. This is the basis for a short 06UTC forecast to T+15hr (to 21UTC).
  • Forecast fields T+03hr to T+09hr from the 06UTC forecast (09UTC to 15UTC) are used as first guess and background fields for 12UTC SCDA.
  • The first guess for the 12UTC 4D-Var LWDA analyses is provided by the analysis from the 12UTC SCDA which has already used all available information with data times between 09UTC and 15UTC (T+03hr and T+09hr from the 06UTC forecast), including incorporation of late arrival data by Continuous Data Assimilation (CoDA).
  • Background fields during the rest of the 4D-Var analysis is provided by T+12hr and T+15hr output from the 06UTC forecast (18UTC and 21UTC).


These 12-hour 4D-Var LWDA analyses propagate information forwards from day to day via the SCDA.


Fig2.6-2: The recurring cycle of the LWDA 4D-Var analysis.  Consider the 12UTC LDWA:  The 00UTC LWDA generates an 00UTC analysis but also provides an 06UTC analysis because it has access to data times between 03UTC and 09UTC.  This 06UTC analysis becomes the basis of an 06UTC short forecast.  This short forecast provides the first guess (at T+3, 09UTC) and then background fields for use by the SCDA 4D-Var process.  The SCDA 12UTC analysis is used as the first guess for use by the LWDA 4D-Var process.  LWDA then uses T+12hr and T+15hr as background fields to the end of the assimilation period (at T+15, 21UTC).  The data arrival cut-off time is 02UTC (14hr after the nominal time (12UTC) of the analysis) in order to ensure all observations with data times between 09UTC and 21UTC are captured.   The cycle is repeated for subsequent analysis cycles 12hr later.


The early delivery assimilation (SCDA)

The early delivery system is also known as the Short Cutoff Daily Assimilation (SCDA).  The system extracts information from the products of the stand-alone LWDA system without waiting for the full LWDA analysis, yet providing sufficient information to enable ECMWF forecasts to be delivered without significant loss of effectiveness.  The early-delivery assimilations do not propagate information from cycle to cycle.  Each analysis is reinitialized with the best available model fields from the LWDA assimilation.

SCDA allows the 00UTC and 12UTC operational analyses to be produced in a timely manner without compromising the quality of the operational forecast products.

The analysis starts 4 hours after the nominal model data time.  However, Continuous Data Assimilation (CoDA) allows any further observations which arrive up until 4hr25min to be included while the analysis process is running (the data is incorporated within the nested analysis loops).  This early cut-off is nevertheless sufficiently late to allow inclusion of more than about 85% of all global observations available within the 6 hour data window centred on the nominal data time of the forecast runs.  Thus:

  • the 00UTC analysis starts at 04:00UTC and uses all observations having a time within the 21-03UTC data window centred on 00UTC and arriving before 04:25UTC.
  • the 12UTC analysis starts at 16:00UTC and uses all observations having a time within the 09-15UTC data window centred on 12UTC and arriving before 16:25UTC.

The first guess and background data for the 4D-Var processes are provided by a forecast T+03hr to T+15hr based on the analysis extracted from the previous (T-6hr) 4D-Var analysis.  Thus:

  • For 00UTC SCDA:
    • A short forecast to T+15hr is run based on the 18UTC analysis derived from the 12UTC 4D-Var LWDA process.  SCDA only needs forecast output to T+9hr from the 18UTC forecast (to 03UTC).
    • The first guess for the 00UTC 4D-Var analyses is provided by the T+3hr output from the 18UTC forecast (21UTC).
    • Background data during the rest of the 4D-Var analysis is provided by T+6hr to T+9hr forecasts from the 18UTC forecast (00UTC and 03UTC).
  • For 12UTC SCDA:
    • A short forecast to T+15hr is run based on the 06UTC analysis derived from the 00UTC 4D-Var LWDA process.  SCDA only needs forecast output to T+9hr from the 06UTC forecast (to 15UTC).
    • The first guess for the 12UTC 4D-Var analyses is provided by the T+3hr output from the 06UTC forecast (09UTC).
    • Background data during the rest of the 4D-Var analysis is provided by T+6hr to T+9hr forecasts from the 06UTC forecast (12UTC and 15UTC).

An EDA process (an ensemble of 4D-Var analyses using data extracted from the previous EDA) is run in parallel with the 4D-Var assimilation.  The output is used to:

  • constrain the 4D-Var assimilation using the EDA background errors output.
  • modify the variability of EDA fields from those produced by the previous EDA run in the light of latest observations (re-centering).
  • create the EDA ensemble of analyses (currently 50).

An ensemble of perturbed analyses is used to create ensemble members (currently 50).

The ENS control (CTRL) is run using the output from the early delivery system analysis (SCDA).   The ensemble is run using the output from the ensemble of perturbed analyses.

The process is repeated at 12 hour intervals.

The early delivery system and associated early cut-off analysis relies on:

  • the first guess and backgrounds supplying about 85% of the value of the 4D-Var analysis (any later observations supply only about 15%).
  • the majority (>85%) of all global observations normally arriving by 4hr:25min after the nominal forecast data time. 

The loss of about 15% of observations arriving after 4hr:25min means the predictive skill of an IFS model is reduced by a few hours but allows dissemination of forecasts some 10 hours earlier.  Thus there is an operational gain of 4 to 6 hours in effective predictability.  It is important to note that the background information always comes from the full 12-hour 4D-Var where almost all available observations have been used.


Fig2.6-3:  The recurring cycle of the early cut-off 4D-var analysis (SCDA).  Consider the 12UTC Early Cut-off (SCDA):  The 00UTC LWDA provides an 06UTC analysis that becomes the basis of an 06UTC short forecast.  This provides the first guess (at T+03, 09UTC); background data during the rest of the 4D-Var analysis is provided by T+6hr to T+9hr forecasts (12UTC and 15UTC) from the 18UTC forecast.  The cut-off time is 16:25UTC (4h25m after the nominal time (12UTC) of the analysis using CoDA between 16:00 and 16:25) to ensure as many as possible observations with data times between 09UTC and 15UTC are captured.  The cycle is repeated for subsequent analysis cycles 12hr later.  The early cut-off 4D-Var (SCDA) 12UTC analysis is used by the ensemble control member (CTRL).  It is also used as first guess for the 12UTC  LWDA.  The EDA runs in parallel with the 4D-Var and a) constrains the 4D-Var analysis and b) provides information for evaluation of perturbations for the ensemble.


Continuous Data Assimilation (CoDA)

It is important to assimilate as much data as possible.  Rejecting observations which arrive after the 4D-Var calculations have started runs the risk of missing the benefit of valuable, though delayed, information.  To capture such delayed data the recently arrived observations are introduced into the 4D-Var assimilation after the calculations have started.  This is done at the beginning of the next available outer loop of the 4D-Var iterations.  The latest possible insertion of delayed data is at the start of the fourth outer loop at about 25 minutes after the assimilation process began.  This allows the analysis to benefit from more recent observations and yields a more accurate final assimilation and analysis. 


Fig2.6-4:  Continuous Data Analysis in use.  Consider the 12UTC Early Cut-off (SCDA):  The SCDA 4D-Var assimilation suite starts at 16:00UTC but, to capture as much data as possible with observation times between 09UTC and 15:00UTC, observations arriving after 16:00UTC  are introduced into the system at the start of each iteration of the algorithm (outer loop within 4D-Var).  In the diagram, trajectory means the analysis as part of the ongoing evolution of the IFS; loop refers to the assimilation cycles within 4D-Var. 


Additional Sources of Information

(Note: In older material there may be references to issues that have subsequently been addressed)