image © AzriSuratmin/iStock/Thinkstock

Rationale of the Forecast Ensemble - ENS

Uncertainties in NWP forecasting

No NWP model can produce consistently or precisely correct forecasts and there must be some uncertainty in the results of each forecast.  The value of NWP forecasts that are produced would be greatly enhanced if the quality of those forecasts could be assessed beforehand.  Consequently methods have been, and continue to be, developed to provide advance knowledge on the certainty (or uncertainty) of a particular forecast, and what possible alternative developments might occur.  This is in parallel with improving the observational network, the data assimilation system, and the NWP models themselves.

The ECMWF forecast ensemble is based upon the idea that incorrect forecasts result from a combination of initial analysis errors and model deficiencies, the former dominating during the first five days or so.

Forecast models can produce incorrect results because of:

  • initial condition uncertainties (also originating from errors in the first guess forecast):
    • Lack of observations.
    • Observation error.
    • Errors in the data assimilation.

  • model uncertainties:
    • Limited resolution.
    • Weaknesses in parameterisation of physical processes.

  • boundary condition uncertainties:
    • Insufficient detail of sub-grid scale orography.
    • Insufficient knowledge of changing surface characteristics.
    • Parameterised derivation of surface fluxes.
       
  • the chaotic nature of the atmosphere:
    • Small uncertainties grow to large errors (unstable flow)
    • Small-scale errors will affect the large-scale (non-linear dynamics)
    • Error-growth is flow dependant.  

But even very good analysis systems and forecast models are prone to errors.

Analysis errors amplify most easily where the the atmosphere is most sensitive to small differences, in particular where strong baroclinic systems develop.  These errors then move downstream and further amplify or change, and thereby affect the large-scale flow.

Structure and operation of the ENS

To estimate the effect of possible initial analysis errors and the consequent uncertainty of the forecasts, an ensemble is formed of many different “perturbed” initial states and one unperturbed analysis (the control member, CTRL).

Currently :

  • Medium range 10 day ensemble consists of 50 members and the unperturbed control member (CTRL).
  • Medium range 15 day ensemble consists of 50 members and the unperturbed control member (CTRL).
  • Extended range 46 day ensemble consists of 100 members and the unperturbed control member CONTROL.
  • Seasonal range 7 month or 13 month ensemble of 50 members and the unperturbed control member CONTROL.

The different perturbations are derived at analysis time during the generation of the ensemble. 

The ensemble forecast suite is then run using each of the perturbed and the unperturbed analyses as a starting point giving a range of forecast results which may diverge radically or remain broadly similar.  To deal with uncertainty in the structure of the parameterisation schemes or with errors due to incomplete IFS modelling of unresolved scales, etc., perturbations are continually inserted into the ensemble members (but not the ensemble control) throughout execution of the forecasts.  The perturbations are supplied by the Stochastically Perturbed Parameterisation Tendencies Scheme (SPPT).   The ensemble control member (CTRL) is unperturbed and does not use these "stochastic physics" perturbations during execution.

Singular Vectors (SVs) pick up localised areas of strong barotropic and baroclinic instability and these are also supplied as perturbations to the ensemble members.

Processing the ensemble of forecasts is computationally expensive. The medium range ENS currently has 50 members and 9km resolution.  But the extended range ENS has 100 members so in order to save computation time the ensemble members are run with a lower resolution, currently 36km.

Qualitative use of the ENS

If the perturbed forecasts more or less agree with the control member forecast, then the atmosphere can be considered to be in a predictable state.  Any of the expected analysis errors appear not have a significant impact.  In such cases it might be possible to issue a categorical forecast with reasonable, but not total, certainty.

If the perturbed forecasts deviate significantly from the control member forecast and from each other, then the atmosphere can be considered to be in a rather unpredictable state.  In such cases it would not be possible to issue a categorical forecast with any certainty. 

The way in which the perturbed forecasts differ from each other provides valuable indications of which weather patterns are likely to develop or, often equally importantly, not develop.  Guidance on how to best to deal with uncertainty and on aids to interpretation of various ensemble outputs are given elsewhere within the this User Guide.


Fig5-1: An ensemble of forecasts produces a range of possible scenarios rather than a single predicted value.  Addition or subtraction of small perturbations to the initial distribution of a parameter give several equally probable ensemble members each slightly different from the initial unperturbed analysis of the control member.  The ensemble members evolve through the forecast period in slightly different ways and the distribution of the ensemble members gives an indication of the likelihood of occurrence of the different scenarios.  In the schematic diagram, compared to initial conditions, a few ensemble members (small peak in ENS numbers) are grouped to give a similar forecast that is cooler, two forecasts predict relatively much warmer conditions, but the majority show a modest warming over initial conditions.  Each solution is possible; the solution associated with the larger peak (larger number of ENS members) is more probable.

Quantitative use of the ENS 

The ensemble mean (EM) forecast, or if required the ensemble median forecast (not necessarily the same as the EM) can be calculated from the ensemble.  This tends to average out the less predictable atmospheric scales.  The accuracy of the EM can be estimated theoretically by the spread of the ensemble so that, on average, the expected EM error is proportional to ensemble spread.  More importantly, the ensemble provides information from which the probability of alternative developments is calculated, in particular those related to risk of extreme or high-impact weather.

The ensemble spread is a measure of the difference between the members and is represented by the standard deviation (Std) with respect to the EM.  On average, small spread indicates high forecast accuracy of the ensemble mean, and indeed of the ensemble members in general; larger spread corresponds to lower forecast accuracy of the ensemble mean, and of most of the ensemble members.  The ensemble spread is flow-dependent and in relative terms will vary for different parameters (e.g. in winter anticyclonic conditions over land spread might be relatively high for 2m temperature, but relatively low for mean sea level pressure). Spread usually increases with the forecast range, but there can be cases when the spread is larger at shorter forecast ranges than at longer ranges.  This might happen when the first days are characterized by strong synoptic systems with complex structures but are followed by large-scale ““fair weather”” high pressure systems.

The spread around the ensemble mean as a measure of accuracy applies only to the ensemble mean forecast error.   It does not apply to the median, nor control members, even if they happen to lie mid-range within the ensemble.  The spread of the ensemble, relative to a particular ensemble member is, for example, about 41% larger than the spread around the ensemble mean.  The spread with respect to the control members is initially the same as for the ensemble mean, but gradually increases, ultimately reaching the same 41% excess as any member (see Fig5-2).

Fig5-2: The diagram shows schematically the relation between the spread of the ensemble for the whole forecast range (orange shaded area). The ensemble mean (red line) lies in the middle of the ensemble spread.  Any individual ensemble member (blue line) can lie anywhere within the spread.  The control member (green line) does not constitute a part of the plume and can even on rare occasions be outside the plume (theoretically on average 4% of the time).

Characteristics of a good ensemble

Forecasts from a good ensemble should: 

  • display no mean errors (bias); otherwise the probabilities will be biased as well.
  • exhibit sharpness (i.e. have relatively small spread where the uncertainty is small).
  • have the ability to span the full climatological range; otherwise the probabilities will either over- or under-forecast the risks of anomalous or extreme weather events.

Systematic errors can be detected by deterministic verification methods (for mean errors) or through probabilistic verification methods (for errors in the variability).

Additional Sources of Information

(Note: In older material there may be references to issues that have subsequently been addressed)