The Anomaly Correlation Coefficient (ACC) is one of the most widely used measures in the verification of spatial fields.   It is the spatial correlation between a forecast anomaly relative to climatology, and a verifying analysis anomaly relative to climatology.  ACC is a measure of how well the forecast anomalies have represented the observed anomalies.  It shows how well the predicted values from a forecast model "fit" with the real-life data.  ACC for a series of forecast lead-times is a measure of how well trends in the predicted anomalies follow trends in actual anomalies.

ACC values lie between +1 and -1.  Where ACC values:

  • approach +1 there is good agreement and the forecast anomaly has had value.
  • lie around 0.5 the forecast errors are similar to those of a forecast based on a climatological average.
  • lie around 0 there is poor agreement and the forecast has had no value.
  • approach –1 the agreement is in anti-phase and the forecast has been very unreliable.

Where the ACC value falls below 0.6 it is considered that the positioning of synoptic scale features ceases to have value for forecasting purposes.   

 

At ECMWF the anomaly correlation coefficient (ACC) scores represent the spacial correlation between:

  • the mean of the anomalies of a forecast product relative to reference model climate and
  • the mean of the anomalies of observations or reanalysis relative to the same reference model climate.


The medium range model climate (M-climate) and extended range model climate (ER-M-climate) are based on re-forecasts spanning the last 20 years, which used the ERA5 reanalysis for their initialisation

The seasonal model climate (S-M-climate) is based on re-forecasts spanning the last 20 years, which used the ERA-interim re-analysis for their initialisation


 In the analysis below (Fig6.2.2-1 and Fig6.2.2-2), forecasts by all IFS models progressively correlate less well with observed values as forecast lead-time increases.  In these graphs HRES has 9Km resolution, the medium range ensemble has 18Km resolution.  The unperturbed ensemble control (CTRL) performs better than any individual perturbed member of ensemble (PF in the diagrams).  The ensemble mean smooths out many smaller scale features and performs best against the verifying analysis.  This supports preferential use of the ensemble mean in practical forecasting.  Typically ACC falls to 0.6 at around day8 or day9 for ensemble control (CTRL), and at around day10 for the ensemble mean (EM).


 Fig6.2.2-1: Anomaly Correlation Coefficients for 850hPa Temperatures. HRES in Red, Ensemble control (CTRL) in Purple, Ensemble mean (EM) in Green, An individual ensemble member (PF) in Cyan.  Note the ACC for ensemble control and HRES are very similar, but the ensemble mean clearly out-performs them and is almost 2½ days better than any individual ensemble member (e.g. ACC of a perturbed member (PF) at Day7 is still being attained by ensemble mean at Day9½).  In these graphs HRES has 9Km resolution, the medium range ensemble has 18Km resolution.  


 Fig6.2.2-2: Anomaly Correlation Coefficients for 500hPa Geopotential.  HRES in Red, Ensemble control (CTRL) in Purple, Ensemble mean (EM) in Green, An individual ensemble member (PF) in Cyan.  The ACC for ensemble control and HRES are very similar.  However, ensemble mean clearly out-performs them and is almost 2 days better than any individual ENS member.  ACC of a perturbed member (PF) at Day7 is still being attained by ensemble mean at Day9).  In these graphs HRES has 9Km resolution, the medium range ensemble has 18Km resolution.  

 

The diagrams above show how the Anomaly Correlation Coefficients vary with forecast lead-time.  The ACC scores have steadily improved over the years at all lead-times.  This reflects the improvement of input data, analysis techniques and IFS model formulation (See Fig6.2.2-3).

 Fig6.2.2-3:  Time series of the annual running mean of anomaly correlations of HRES 500 hPa height forecasts evaluated against the operational analyses for the period 2000 until 2022.  Values are running 12month average scores.  Forecast lead-times in days ahead - 3(blue), 5(red), 7(green) and 10(yellow) - are shown for scores averaged over the northern extra-tropics (bold lines) and southern extra-tropics (thin lines).  The shading shows differences in scores between the two hemispheres at the forecast ranges indicated.   Currently 3-day, 5-day, 7-day, 10-day forecasts have attained approximately 98.5%, 92%, 80%, 50% anomaly correlation (ACC) respectively.



Current Anomaly Correlation Coefficient (ACC) scores for northern hemisphere, Southern Hemisphere, tropics and Europe are available.

The score for the northern hemisphere extra-tropics is a primary headline score of the ECMWF high resolution forecast (HRES). 

Fig6.2.2-4: The plot shows the forecast lead-time (in days) at which Anomaly Correlation Coefficient (ACC) of the HRES forecast dropped below 80% for:

  • the month mean (blue line, with blue spots at each month),
  • twelve-month mean centred on that month (red line).

Verification follows updated WMO/CBS guidelines as specified in the Manual on the GDPFS, Volume 1, Part II, Attachment II.7, Table F, (2010 Edition - Updated in 2012).