EFAS v4.0 medium-range forecast skill is evaluated using the Continuous Ranked Probability Skill Score (CRPSS) for ECMWF-ENS 20-year reforecasts against a persistence benchmark forecast with respect to EFAS v4.0 forced simulations (sfo) as proxy observations.  Further details on the evaluation methodology can be found here: EFAS medium-range forecast skill.

Forecast skill by lead time

Results show that EFAS v4.0 (6 hourly) is skilful (i.e. CRPSS > 0) against a persistence benchmark forecast in virtually all 2651 stations tested for lead times up to 10 days (Figure 1). Skill is highest for sub-daily lead times and decays as a function of lead time. The CRPSS for a 6 hour lead time is 0.92 with an interquartile range of 0.79 to 0.98. At a lead time of 1 day, the CRPSS is 0.69 (0.56, 0.85), reducing to 0.64 (0.57, 0.71), 0.62 (0.56, 0.68) and 0.53 (0.49, 0.57) by lead time 3-, 5-, and 10-days, respectively. 

Figure 1: EFAS v4.0 (forced with ECMWF-ENS) forecast skill across Europe from the Continuous Ranked Probability Skill Score (CRPSS) against a persistence benchmark forecast with respect to EFAS v4.0 forced simulations (sfo) for n=2651 fixed reporting point stations. The solid blue line is the European-wide median CRPSS with blue shaded bands representing the the 25th to 75th and 5th to 95th percentile ranges. 

Spatial distribution of forecast skill

The spatial distribution of forecast skill is shown in Figure 2 for 1-, 3-, 5-, and 10-day lead times. Highest forecast skill is found for stations in central and north eastern Europe. Skill is lower in Iceland, Norway, north west UK, southern Spain as well as areas in the Alps and bordering the Black Sea. Forecast skill is positively correlated with catchment area (Figure 3), with a Pearson correlation coefficient = 0.65 for log(catchment area) and CRPSS at a 1 day lead time. Larger catchments have higher forecast skill. There are 12 catchments with negative skill at lead time = 1 day (i.e. CRPSS < 0), all with catchment areas < 3,500 km². 

These initial results point towards lowest forecast skill in smaller upland catchments as well those with large amounts of snow and/or ice. These types of conditions are known to be particularly challenging for hydrological forecasting systems and results presented here will help guide areas for further research and development to improve forecast skill in these situations. 

Figure 2: Distribution of EFAS v4.0 (forced with ECMWF-ENS) forecast skill across the EFAS domain from the Continuous Ranked Probability Skill Score (CRPSS) against a persistence benchmark forecast with respect to EFAS v4.0 forced simulations (sfo) for n=2651 fixed reporting point stations. Points with darker shades of blue (red) have more positive (negative) skill. 

Figure 3: Catchment area (km²) in log scale against the Continuous Ranked Probability Skill Score (CRPSS) at a 1 day lead time for n=2651 fixed reporting point stations.