The 0.05 degrees quasi-global (-180,180,90,-60) implementation of the LISFLOOD model, was calibrated using 1995 in-situ discharge gauge stations with at least 4-years-long time series of measurements more recent than 01 January 1980.

Firstly, the Distributed Evolutionary Algorithm for Python (DEAP, Fortin et al. 2012¹) was used to optimize the parameters of catchments for which discharge data were available (gauged catchments). Secondly, a pragmatic regionalization approach was implemented to transfer the parameters from the gauged catchments (donors) to the ungauged catchments. The modified Kling Gupta Efficiency (Gupta et al., 2009²) was selected as objective function and a minimum drainage area of 500 km² was used for both the above explained steps of the calibration. The combined calibration approach delivered 14 parameter maps with quasi-global extent.

The calibrated parameter maps were used to execute the long-term run (LTR), a continuous simulation with forced with C3S ERA5 reanalysis meteorological data, for the period 01/01/1979-31/12/2019. Simulated daily discharge time series were then compared against observed discharge from the 1995 calibration stations (the first three years are generally excluded from evaluation; the exceptions are explained here GloFAS v4.0 calibration data - Copernicus Emergency Management Service - CEMS - ECMWF Confluence Wiki). Observation time series with different length and different temporal coverage were used for calibration in the semi-global domain. For this reason, hydrological modelling performance is evaluated using all available discharge data rather than on calibration and verification periods separately. This page summarises GloFAS v4 hydrological skill.

Overview

The hydrological performance of GloFAS v4 is expressed by the modified Kling-Gupta Efficiency (KGE') (Knoben et al., 2019³). A detailed explanation of the modified Kling-Gupta Efficiency (KGE') is available from this page.

Figure 1 shows the cumulative distribution function of KGE' values, as well as the KGE' distribution for the 1995 calibration stations. The median KGE' is 0.70, with the calibrated parameters leading to higher accuracy than the mean flow benchmark (i.e. KGE’ > -0.41, Knoben et al., 2019³) for 92.9% of the gauged catchments. 

Figure 1 –  KGE' histogram (blue bars) distribution and empirical cumulative distribution function(red line) for all the 1995 calibration stations of GloFAS v4 and all the available data. The green line shows the optimal performance.

Figure 2 presents the results of GloFAS v4 for the 1995 calibration points in terms of KGE' components: linear correlation between observations and simulations, bias, and a measure of the flow variability error (Knoben et al., 2019³). 

Figure 2 –  KGE' components (correlation, bias, variability) histogram distribution (violet bars) and empirical cumulative distribution function 9red line) for all the 1995 calibration stations of GloFAS v4 and all the available data. The green lines show the optimal performance.

Spatial analysis

Figure 3 shows the spatial distribution of KGE' values for the calibration stations. KGE' values > 0.7 are shown in light blue and blue. KGE’ values < -0.41 are shown in black. KGE' is generally uniformly distributed across the domain, with higher performance (light blue and blue) in large parts of North and South America, Central Europe, and Asia. Calibrated catchments with high performances are also found in Africa and Oceania. The lowest performances (black) are often concentrated in catchments with strongly regulated rivers.

Figure 3 –  Spatial distribution of the hydrological performance (KGE') of GloFAS v4 across the domain for the 1995 calibration stations and all the available data.

A low score during evaluation of LISFLOOD OS model calibration is not necessarily an indicator for decreased forecast performance of the global flood awareness system. GloFAS forecasts are compared to model derived thresholds (Thielen et al., 2009⁴; Bartholmes et al., 2009⁵), this comparison eliminates systematic bias. In some calibration stations, the systematic bias leads to an overall lower score in hydrological performance. Nevertheless, correlation is a desired quality in hydrological performance as it represents the timing of flood peaks. Given the mathematical structure of KGE', all stations where KGE'>=0.7 have correlation >=0.7 (Gupta et al., 2009²). Conversely, some of the stations with KGE'< 0.7 can have correlation>= 0.7; but associated to a large mean bias and/or variability bias. Calibration points with low KGE' but correlation >=0.7 won't decrease the forecast performance of the Global Flood Awareness System, even if forecast discharge will exhibit large bias. Figure 4 shows a combination of the spatial distribution of GloFAS KGE' and correlation. Stations with KGE'<0.7 and Correlation>=0.7 are highlighted in white. Compared to Figure 3, 336 calibration stations with KGE<0.7 show a Correlation>0.7: these stations are represented in white in Figure 4.

Figure 4 –  Spatial distribution of the hydrological performance (KGE') of GloFAS v4 across the domain combined with correlation: stations with KGE'<0.7  and correlation>=0.7 are highlighted in white.

Figures 5, 6, and 7 present the spatial distribution of GloFAS v4 hydrological performance across the quasi-global domain in terms of KGE' components: correlation, mean bias and variability bias.

Figure 5 –  Spatial distribution of correlation of GloFAS v4 at all 1995 calibration stations (evaluated using all the available data).

Figure 6 –  Spatial distribution of bias of GloFAS v4 at all 1995 calibration stations (evaluated using all the available data).

Figure 7 –  Spatial distribution of variability of GloFAS v4 at all 1995 calibration stations (evaluated using all the available data).

Comparison of GloFASv4 against GloFASv3: overview

GloFAS v4 is the first GloFAS version with 0.05 degrees resolution, therefore allowing the representation of hydrological processes with 4 times higher resolution than all the previous GloFAS versions. Moreover, compared to the former 0.1 degrees resolution set-up, the 0.05 resolution set-up was developed by making use of the latest research findings, remote sensing and in-situ data collections (CEMS-Flood surface field dataset - Copernicus Emergency Management Service - CEMS - ECMWF Confluence Wiki). These significant differences in the model set ups hinder a quantitative comparison between GloFAS v4 and GloFAS v3. However, an attempt was made to show the improvements of the new GloFAS v4 compared to GloFAS v3 (GloFAS v3 implementation set-up and calibration are described into detail by Alfieri et al. 2020⁶).

Out of the 1995 calibration stations used in GloFAS v4, only 1173 could be used for the comparison with GloFAS v3 calibration. GloFAS v3 used 1226 calibration stations (Alfieri et al. 2020⁶), however, 53 of those stations were not included in the calibration of GloFAS v4 because they were replaced by near-by stations with longer and higher quality time series of observations.

It is important to note that calibration periods in the two versions could be different, therefore the KGE' were computed using observed daily discharge for the entire observation period available to GloFAS v4 calibration.

Figure 8 shows the KGE' cumulative distribution functions for the 1173 shared stations: GloFAS v4 in red and GloFAS v3 in black. The entirely revamped, high resolution model set-up and new calibration of GloFAS v4 allowed an increase of the percentage of stations with KGE' > 0.7 from 38% (GloFAS v3) to 58% (GloFAS v4). Moreover, the percentage of stations for which calibrated parameters lead to lower accuracy than the mean flow benchmark (KGE’<-0.41) decreased from 13.6% (GloFAS v3) to 2.8% (GloFAS v4).

Figure 8  - KGE' Cumulative distribution function for GloFAS v3 (black) and GloFAS v4 (red) 

Comparison of GloFAS v4 against GloFAS v3: spatial analysis

Figure 9 presents the spatial distribution of KGE' skill score between GloFAS v4 and GloFAS v3 (benchmark) for the 1173 common stations (and all the available data). Improvements are represented in green and cyan, substantially similar values with +- 0.05 in KGE' skill score are in white, degradations are represented in orange and red. 

Figure 9 - Spatial distribution of KGE' skill score between GloFAS v4 and GloFAS v3 (benchmark).

Figure 10 presents the spatial distribution of difference between GloFAS v4 KGE’ and GloFAS v3 KGE’(benchmark) for the 1173 common stations. Improvements are represented in green and blue, substantially similar values in white, degradations are represented in orange and red.

Figure 10 - Spatial distribution of KGE' difference between GloFAS v4 and GloFAS v3 (benchmark).

References

¹ Fortin, F. A., De Rainville, F. M., Gardner, M. A. G., Parizeau, M., & Gagné, C. (2012). DEAP: Evolutionary algorithms made easy. The Journal of Machine Learning Research, 13(1), 2171-2175.  https://jmlr.org/papers/volume13/fortin12a/fortin12a.pdf

² Gupta, H. V., Kling, H., Yilmaz, K. K., & Martinez, G. F. (2009). Decomposition of the mean squared error and NSE performance criteria: Implications for improving hydrological modelling. Journal of hydrology, 377(1-2), 80-91. https://www.sciencedirect.com/science/article/pii/S0022169409004843?via%3Dihub

³ Knoben, W. J. M., Freer, J. E., and Woods, R. A (2019). Technical note: Inherent benchmark or not? Comparing Nash–Sutcliffe and Kling–Gupta efficiency scores, Hydrol. Earth Syst. Sci., 23, 4323–4331, https://doi.org/10.5194/hess-23-4323-2019.

⁴ Thielen, J., Bartholmes, J., Ramos, M.-H., and de Roo, A. (2009). The European Flood Alert System – Part 1: Concept and development, Hydrol. Earth Syst. Sci., 13, 125–140, https://doi.org/10.5194/hess-13-125-2009.

⁵ Bartholmes, J. C., Thielen, J., Ramos, M. H., and Gentilini, S. (2009). The European flood alert system EFAS – Part 2: Statistical skill assessment of probabilistic and deterministic operational forecasts, Hydrol. Earth Syst. Sci., 13, 141–153, https://doi.org/10.5194/hess-13-141-2009.

⁶ Alfieri, L., Lorini, V., Hirpa, F.A., Harrigan, S., Zsoter, E., Prudhomme, C., & Salamon, P. (2020). A global streamflow reanalysis for 1980–2018. Journal of Hydrology X, 6., https://doi.org/10.1016/j.hydroa.2019.100049.