Comparison of scores of IFS Cycle 46r1 (combination of research experiments and the current test suite) and IFS Cycle 45r1 (the current operational cycle) verified by the respective analyses or in-situ observations (TEMP, SYNOP, BUOY) at 00 and 12UTC in the period 1 June 2017 to 14 May 2019 (in total 645  high-resolution and 312 ensemble forecast runs).

Each symbol in a cell represents 1 forecast day, i.e 10 days for HRES and 15 days for ENS.

HRES


Extratropical
Northern
Hemisphere
Extratropical
Southern
Hemisphere
Tropics

Anomaly correlation/
SEEPS(*)
RMS error/
Std. dev.of error(*)
Anomaly correlation/
SEEPS(*)
RMS error/
Std. dev.of error(*)
Anomaly correlation/
SEEPS(*)
RMS error/
Std. dev.of error(*)
Analysis




















Geopotential


100



250



500



850



Mean sea level pressure



Temperature



100

250

500

850

1000

2m temperature

Wind



100

250

500

850

1000

10m wind

Relative humidity250


700

10m wind at sea

Significant wave height

(*)

(*)

(*)

Mean wave period

(*)

(*)

(*)

Observations

















Geopotential


100



250



500



850



Temperature


100

250

500

850

2m temperature



Wind


100

250

500

850

10m wind



Relative humidity250


700

2m dew-point



Total cloud cover



24h precipitation

(*)

(*)

(*)


EuropeNorthern AmericaEast Asia

Anomaly correlation/
SEEPS(*)
RMS error/
Std. dev.of error(*)
Anomaly correlation/
SEEPS(*)
RMS error/
Std. dev.of error(*)
Anomaly correlation/
SEEPS(*)
RMS error/
Std. dev.of error(*)
Analysis

















Geopotential


100

250

500

850

Mean sea level pressure

Temperature



100

250

500

850

1000

2m temperature

Wind



100

250

500

850

1000

10m wind

Relative
humidity
250


700

Observations

















Geopotential


100

250

500

850

Temperature


100

250

500

850

2m temperature



Wind


100

250

500

850

10m wind



Relative
humidity
250


700

2m dew-point



Total cloud cover



24h precipitation

(*)

(*)

(*)


Northern AtlanticNorthern Pacific

Anomaly correlation/
SEEPS(*)
RMS error/
Std. dev.of error(*)
Anomaly correlation/
SEEPS(*)
RMS error/
Std. dev.of error(*)
AnalysisGeopotential


100


250


500


850


Mean sea level pressure


Temperature



100


250


500


850


1000


2m temperature


Wind



100


250


500


850


1000


10m wind


Relative
humidity
250



700


10m wind at sea


Significant wave height

(*)

(*)


Mean wave period

(*)

(*)


ArcticAntarctic

Anomaly correlation/
SEEPS(*)
RMS error/
Std. dev.of error(*)
Anomaly correlation/
SEEPS(*)
RMS error/
Std. dev.of error(*)
Analysis

















Geopotential


100

250

500

850

Mean sea level pressure

Temperature



100

250

500

850

1000

2m temperature

Wind



100

250

500

850

1000

10m wind

Relative
humidity
250


700

Observations

















Geopotential


100

250

500

850

Temperature


100

250

500

850

2m temperature


Wind


100

250

500

850

10m wind


Relative
humidity
250


700

2m dew-point


Total cloud cover


24h precipitation

(*)

(*)

ENS



Extratropical
Northern
Hemisphere
Extratropical
Southern
Hemisphere
Tropics

EM RMS errorCRPSEM RMS errorCRPSEM RMS errorCRPS
AnalysisGeopotential100





250





500





850




Mean sea level pressure




Temperature100



250



500



850


Wind speed100



250



500



850


Relative
humidity
200



700


2m temperature


10m wind at sea


Significant wave height


Mean wave period

ObservationsGeopotential100





250





500





850




Temperature100



250



500



850


Wind speed100



250



500



850


Relative
humidity
200



700


2m temperature


2m dew-point


Total cloud cover


10m wind


24h precipitation


EuropeNorthern AmericaEast Asia

EM RMS errorCRPSEM RMS errorCRPSEM RMS errorCRPS
AnalysisGeopotential100



250



500



850


Mean sea level pressure


Temperature100



250



500



850


Wind speed100



250



500



850


Relative
humidity
200



700


2m temperature

ObservationsGeopotential100



250



500



850


Temperature100



250



500



850


Wind speed100



250



500



850


Relative humidity200



700


2m temperature


2m dew-point


Total cloud cover


10m wind


24h precipitation


ArcticAntarctic

EM RMS errorCRPSEM RMS errorCRPS
Analysis














Geopotential


100

250

500

850

Mean sea level pressure

Temperature100


250


500


850

Wind speed100


250


500


850

Relative
humidity
200


700

2m temperature

Observations

















Geopotential


100

250

500

850

Temperature


100

250

500

850

Wind speed100


250


500


850

Relative
humidity
200


700

2m temperature

2m dew-point

Total cloud cover

10m wind

24h precipitation

Symbol legend: for a given forecast step...
46r1 better than 45r1 statistically significant with 99.7% confidence
46r1 better than 45r1 statistically significant with 95% confidence
46r1 better than 45r1 statistically significant with 68% confidence
not really any difference between 45r1 and 46r1
46r1 worse than 45r1 statistically significant with 68% confidence
46r1 worse than 45r1 statistically significant with 95% confidence
46r1 worse than 45r1 statistically significant with 99.7% confidence

2 Comments

  1. What is the situation for the stratosphere? Why are results not shown for at least one or two levels with pressures lower than 100hPa?

    1. We see statistically significant improvements in temperature and vector wind in the stratosphere. The signal is most pronounced between 100 hPa and 10 hPa, and it is present between 10 hPa and 1 hPa at short lead times. In the scorecard, emphasis has usually been on the troposphere and (extratropical) lower stratosphere, due to their relevance for most users. We will take your suggestion into account for future cycles.