The 1 arcmin (0.01667 degrees) pan-European implementation of the LISFLOOD model, was calibrated using 1903 in-situ discharge gauge stations with at least 4-years-long time series of measurements more recent than 01 January 1990.

Firstly, the Distributed Evolutionary Algorithm for Python (DEAP, Fortin et al. 2012¹) was used to optimise the parameters of catchments for which discharge data were available (gauged catchments). Secondly, a pragmatic regionalisation approach was implemented to transfer the parameters from the gauged catchments (donors) to the ungauged catchments. The modified Kling Gupta Efficiency (Gupta et al., 2009²) was selected as objective function and a minimum drainage area of 150 km² was used for both the above explained steps of the calibration. The combined calibration approach delivered 14 parameter maps with pan-European extent.

The calibrated parameter maps were used to execute the long-term run (LTR), a continuous simulation with model forced with meteorological observation maps, for the period 01/01/1990-31/12/2021 (the first three years are generally excluded from evaluation, the exceptions are explained here EFAS v5.0 calibration data - Copernicus Emergency Management Service - CEMS - ECMWF Confluence Wiki). Simulated 6-hourly discharge time series were then compared against observed discharge from the 1903 calibration stations. Observation time series with different length and different temporal coverage were used for calibration in the pan-European domain. For this reason, hydrological modelling performance is evaluated using all available discharge data rather than on calibration and verification periods separately. This page summarises EFAS v5 hydrological skill.

Overview

The hydrological performance of EFAS v5 is expressed by the modified Kling-Gupta Efficiency (KGE') (Knoben et al. 2019). A detailed explanation of the modified Kling-Gupta Efficiency (KGE') is available here.

Figure 1 shows the cumulative distribution function of KGE' values of EFAS v5, as well as the KGE' distribution for the 1903 calibration stations. The median KGE' is 0.694, with the calibrated parameters leading to higher accuracy than the mean flow benchmark (i.e. KGE’ > -0.41, Knoben et al. 2019) for 99.0% of the gauged catchments. 

Figure 1 –  EFAS v5 KGE' histogram (blue bars) distribution and empirical cumulative distribution function (red line) for all the 1903 calibration stations and all the available discharge observations. The green line shows the optimal performance. The dashed blue line shows KGE' =-0.41.

Figure 2 presents the results of EFAS v5 for the 1903 calibration points in terms of KGE' components: linear correlation between observations and simulations, bias, and a measure of the flow variability error (Knoben et al., 2019). 

Figure 2 –  KGE' components (correlation, bias, variability) histogram distribution (blue bars) and empirical cumulative distribution function (red line) for all the 1903 calibration stations used in EFAS v5 and all the available discharge observations. The green lines show the optimal performance.

Spatial analysis

Figure 3 shows the spatial distribution of KGE' values of EFAS v5 for the calibration stations. KGE' values > 0.7 are shown in light blue and blue. KGE’ values < -0.41 are shown in green. KGE' is generally uniformly distributed across the domain, with higher performance (light blue and blue) in large parts of central Europe. The lowest performances (green) are often concentrated in catchments with strongly regulated rivers.

Figure 3 –  Spatial distribution of the hydrological performance (KGE') of EFAS v5 across the domain for the 1903 calibration stations and all the available discharge observations.

A low score during evaluation of LISFLOOD OS model calibration is not necessarily an indicator for decreased forecast performance of the European flood awareness system. EFAS forecasts are compared to model derived thresholds (Thielen et al., 2009; Bartholmes et al., 2009), this comparison eliminates systematic bias. In some calibration stations, the systematic bias leads to an overall lower score in hydrological performance. Nevertheless, correlation is a desired quality in hydrological performance as it represents the timing of flood peaks. Given the mathematical structure of KGE', all stations where KGE'>=0.7 have correlation >=0.7 (Gupta et al., 2009). Conversely, some of the stations with KGE'< 0.7 can have correlation>0.7; but associated to a large mean bias and/or variability bias. Calibration points with low KGE' but correlation >=0.7 won't decrease the forecast performance of the European Flood Awareness System, even if forecast discharge will exhibit large bias. Figure 4 shows a combination of the spatial distribution of EFAS v5 KGE' and correlation. Stations with KGE'<0.7 and Correlation>=0.7 are highlighted in cyan. Compared to Figure 3, 387 calibration stations with KGE<0.7 show a Correlation>0.7: these stations are represented in cyan in Figure 4.

Figure 4 –  Spatial distribution of the hydrological performance (KGE') of EFAS v5 across the domain combined with correlation: stations with KGE'<0.7  and correlation>=0.7 are highlighted in cyan.

Figures 5, 6, and 7 present the spatial distribution of EFAS v5 hydrological performance across the pan-European domain in terms of KGE' components: correlation, mean bias and variability bias.

Figure 5 –  Spatial distribution of correlation at all 1903 calibration stations used for EFAS v5 (evaluated using all the available discharge observations).

Figure 6 –  Spatial distribution of bias at all 1903 calibration stations used for EFAS v5 (evaluated using all the available discharge observations).

Figure 7 –  Spatial distribution of variability at all 1903 calibration stations used for EFAS v5 (evaluated using all the available discharge observations).

Comparison of EFASv5 against EFASv4: overview

EFAS v5 is the first EFAS version using 1 arcmin (0.016667 degrees) resolution, therefore allowing the representation of hydrological processes with 4 times higher resolution than the previous EFAS versions. Moreover, compared to the former 5km resolution set-up, the 1 arcmin (0.016667 degrees) resolution set-up was developed by making use of the latest research findings. These significant differences in the model set ups hinder a quantitative comparison between EFAS v5 and EFAS v4. However, an attempt was made to show the improvements of the new EFAS v5 compared to EFAS v4.

Figure 8 shows the KGE' cumulative distribution functions for the 1903 calibration stations used in EFAS v5 and the 1133 calibration stations used in EFAS v4: EFAS v5 in red and EFAS v4 in black. In EFAS v5. the entirely revamped, high resolution model set-up and new calibration allowed an increase from 43% to 49% in the percentage of stations with KGE' > 0.7. Moreover, the percentage of stations for which calibrated parameters lead to lower accuracy than the mean flow benchmark (KGE’<-0.41) decreased from 5.6% to 1.01%.

Figure 8  - KGE' Cumulative distribution function for EFAS v5 (red) and EFAS v4 (black) 

Out of the 1903 calibration stations used in EFAS v5, only 1035 are in common with EFAS v4 calibration. However, 1647 EFAS v5 stations where directly used as calibration station in EFAS v4 (1035) or belong to the EFAS v4 calibrated area. Figure 9 shows the KGE' cumulative distribution functions for the 1647 EFAS v5 calibration stations that belong to EFAS 4 calibration area: EFAS v5 in red and EFAS v4 in black. Figure 10 shows the KGE' cumulative distribution functions for the EFAS v5 calibration stations that belong to EFAS 4 and have an upstream drainage area smaller than 500 Km2.

Figure 9 - KGE' Cumulative distribution function for EFAS v5 (red) and EFAS v4 (black) for the EFAS v5 stations belonging to the EFAS v4 calibrated area. Right panel shows the stations with upstream drained area smaller than 500 km2. 

It is important to note that calibration periods in the two versions could be different, therefore the KGE' were computed using observed discharge for the entire observation period available to EFASv5 calibration.

Comparison of EFASv5 against EFASv4: spatial analysis

Figure 10 presents the spatial distribution of difference between EFAS v5 KGE’ and EFAS v4 KGE’ (benchmark) for the 1657 calibration stations in EFAS v5 that belong to EFAS v4 calibrated area. Improvements are represented in green and blue, substantially similar values in white , degradations are represented in orange and red.

Figure 10 - Spatial distribution of KGE' difference between EFAS v5 and EFAS v4 (benchmark).