Anomaly Correlation Coefficient
ECMWF primary headline score for HRES
Anomaly Correlation Coefficient gives a measure of the effectiveness of the model by showing when the ACC of 500 hPa height falls below a threshold. ACC is one of the most widely used measures in the verification of spacial fields.
The Anomaly Correlation Coefficient is the spacial correlation between the forecast anomaly and the verifying analysis anomaly where the anomalies are computed with respect to a model climate (M-climate, ER-M-climate, S-M-climate). Plots show the lead-time that ACC falls to a given skill threshold.
High ACC indicates good effectiveness of the model.
Current Anomaly Correlation Coefficient diagram.
Continuous ranked probability score (CRPS)
ECMWF primary headline score for medium range
The 12-month running mean percentage of continuous ranked probability score (CRPS) values for 2 m temperature exceeding 5 K at day 5 in the extra-tropics (poleward of 30° latitude), verified against SYNOP observations.
The Continuous Ranked Probability Skill Score (CRPSS) is a measure of how good forecasts are in matching observed outcomes. Where:
- CRPSS = 1 the forecast has perfect skill compared to climatology.
- CRPSS = 0 the forecast has no skill compared to climatology.
- CRPSS = a negative value the forecast is less accurate than climatology.
Discrete Ranked Probability Skill Score (RPSS-D)
ECMWF Primary headline score for extended range
The de-biased Discrete Ranked Probability Skill Score (RPSS-D) for terciles of the mean 2 metre temperature in the northern extra‐tropics in week 3 of the forecast (days 15‐21).
The score is based on the evaluation of re-forecasts against SYNOP observations. Discrete Ranked Probability Skill Score (RPSS) is the deviation of the forecast values placed within a category against corresponding observations that actually lie within that category (e.g. tercile, quintile, etc.) when compared with model climate (M-climate). The words "discrete" and "ranked" refer to the discrete nature of the ranks or categories.
Where:
- RPSS = 1 the forecast has perfect skill compared to the reference (observations, analyses or climatology) - forecast beneficial;
- RPSS = 0 the forecast has no skill compared to the reference (observations, analyses or climatology) - forecast has no benefit over climatology;
- RPSS = a negative value the forecast is less accurate than the reference (observations, analyses or climatology) - forecast misleading.
Stable equity error in probability space (1-SEEPS)
ECMWF supplementary headline score
SEEPS gives a measure of the effectiveness of the model by showing when the 1-SEEPS of 24h precipitation falls to a threshold.
Stable Equitable Error in Probability Space Score (SEEPS) assesses the performance of the precipitation forecast but additionally takes account of climate differences between stations by evaluating the forecast precipitation against the salient features of the local weather. Forecast precipitation accumulated over 24 hours is evaluated against observed precipitation amounts reported from SYNOP stations. At each observation location, the weather is partitioned into three categories: “dry”, “light precipitation” and “heavy precipitation”. The boundary between “light” and “heavy” is determined by the station climatology so that SEEPS assesses salient features of the local weather and accounts for climate differences between stations. The SEEPS score evaluates the performance of the forecast across all three categories.
The plot shows the range of forecast lead-time (in days) at which the 24 hour precipitation forecast 1-SEEPS score falls below 0.45 for:
- 365-day mean centred on each month (red line).
This score for the extra-tropics is a supplementary headline score for the ECMWF HRES.
EFI ROC skill (10m wind at day4)
ECMWF supplementary headline score
Extreme Forecast Index (EFI) is verified against analysis where an extreme event is taken as an observation exceeding 95th percentile of station climate.
Current EFI ROC skill diagram.
Fraction of large CRPS (2m temperature)
ECMWF complementary headline score
The proportion of forecasts at which the Continuous Ranked Probability Score (CRPS) of the 2 metre temperature forecast from the ECMWF ensemble (ENS) at a lead time of 120 hours exceeded 5°C.
The CRPS for the forecast temperature is calculated comparing the Cumulative Distribution Functions (CDF) for the forecast temperature against observations (or analyses) of temperature over a given period and here has the dimensions of temperature.
Overview
Community Forums
Content Tools