Contributors: B. Calmettes (CLS), G. Calassou (CLS), N. Taburet (CLS), L. Carrea (University of Reading), C.J. Merchant (University of Reading)

Issued by: CLS / B. Calmettes

Date: 19/07/2023

Ref: C3S2_312a_Lot4.WP2-FDDP-LK-v1_202212_LWL_PQAR-v4_i1.1

Official reference number service contract: 2021/C3S2_312a_Lot4_EODC/SC1

History of modifications

List of datasets covered by this document

Related documents 

Acronyms

General definitions

Accuracy: The closeness between the measured value and the true quantity value.

Precision: Closeness between measured values obtained by replicate measurements on the same object under similar conditions.

Bias: Estimate of a systematic error.

Lake Water Level: Measure of the absolute height of the reflecting lake water surface beneath the satellite with respect to a vertical datum (geoid) and expressed in metres.

Dispersion: Describes the degree of variation of successive measurements. This performance indicator provides information about the precision of the estimated data.

High-frequency variations: variation of the high frequency signal because of errors due to models or bias.

Mean time step: Average time between two valid measures.

Scope of the document

This document is the Product Quality Assessment Report (PQAR) for the Copernicus Climate Change Service (C3S) Lake Water Level product. This document presents results of the quality assessment for the provided datasets according to the validation methods and strategies described in the Product Quality Assurance Document [D4].

Executive summary

The C3S Lake production system (C3S ECV LK) provides an operational service, generating lake surface water temperature (LSWT) and lake water level (LWL) climate datasets for a wide variety of users within the climate change community. The present document covers the LWL level system.

The quality assessment analysis for the lake water level product consisted of two distinct parts: i) assessing the absolute error with the validation of the data, and (ii) assessing the relative error estimate by the comparison of generated products with external data. Quantifying absolute error was performed by analysing the error generated by the instruments and processing over time. The C3S lake water level product is based on measurements from several altimetry missions, with technology that has developed and improved in consecutive missions (going from standard altimeters as Low Resolution Mode onboard of Jason-3 to Synthetic Aperture Radar - SAR onboard of Sentinel-6A). Estimating relative error is achieved by comparing the generated products with external data from either i) other altimetry-based products or (ii) products derived from in-situ measurements.

This document presents the results of the quality assessments undertaken for the product. The product validation and methodology are described in Section 1, including i) a description of the lake water level product being analysed and (ii) the description of the different external products used for its validation. Section 2 details the validation results, both absolute and relative assessment. Section 3 concerns applications specific assessments, however there are none at the moment. Section 4 contains the summary to the compliance of the generated lake water level dataset with respect to the user requirements. The complete tables and figures for all the in-situ validation tests are shown in annexes.

1. Product validation methodology

1.1. Validated products

The Water Level is the measure of the absolute height of the reflecting water surface beneath the satellite with respect to a vertical datum (geoid) and expressed in metres. The C3S lakes products comprise a long-term climate data record (CDR). The time series has been computed from multiple altimetry satellites since late 1992 to 2022 inclusive. The time periods used for each satellite/instrument are provided in Table 1 but may vary from one lake to the other, depending on the orbits of the satellites with respect to the location of the lake.

Table 1: Time periods for the satellite/instrument used to generate the lake product.

A detailed description of the product generation is provided in the Algorithm Theoretical Basis Document (ATBD) [D3] with further information on the product given in the Product User Guide and Specifications (PUGS) [D5].

1.2. Validating datasets

A combination of in-situ and external altimetry-based products are used to assess the quality of the C3S lakes products. The list of datasets used is provided in Table 2.

Table 2: Datasets used in the assessment of the LWL data product split by altimetry-based and in-situ data.

1.3. Description of product validation methodology

The quality assessment of the LWL product involved the comparison of the dataset to external data (in-situ and altimetry-based), as well as tests to determine the long-term stability of the product at a climate scale. An overview of the methodology is presented here. However, for a more comprehensive description, please see the associated Product Quality Assurance Document (PQAD; [D4]).

1.3.1. Absolute error assessment

Altimeters measure the distance between the satellite and lake surface, with numerous processing steps needed to derive accurate estimates of this distance from the radar signal. The uncertainties or errors in the LWL products are induced by two categories of errors: measurement errors and processing errors.

Measurement errors may have several causes. Numerous influences on the radar signal should be considered, and corrections need to be applied to take into account various physical phenomena (for further information see the ATBD [D3]). Some are already evaluated as the geoid, the ionospheric correction, wet and dry tropospheric corrections and earth and polar tides. However, for other phenomena, it is not possible to correct for these effects because the information (the corrections) is not currently available operationally at global scale (such as for wind effects or “oceanic” tides in large lakes), even though this may induce an uncertainty of a few centimetres. Processing errors are linked to the estimation of parameter files containing, for each lake, intermission and inter track biases and the estimated maximum variation in the water level.

For inland waters, the dominant source of measurement errors is land contamination of the footprint: in some configurations. Nearby land may be as echogenic as water and interfere with the radar echo. In this case, the range measurement, hence the water level measurement, may be affected.  The main challenge of the deriving viable LWL estimates is therefore to correctly identify the valid measurements.

Additionally, the Lake Water Level products may contain altimeter data from multiple satellites tracks as well as different missions. Transects (intersections between satellite tracks and lakes) are on average longer on large lakes. Since the land contamination of the footprint is the major source of error in the measurements, transects on large lakes have both a higher number and a higher percentage of measurements that are not contaminated by this type of errors. The precision is thus better for large lakes.  For lakes assessment, the precision of the measurements provides reliable information on lake water variation, whereas accuracy refers to a relative measurement, based on the datum used as reference.

Three performance indicators have been chosen to assess the quality of lake products in terms of their absolute error:

• Dispersion: This metric quantifies the dispersion of the individual successive measurements recorded by the altimeter when flying above the lake at a given time. It thus quantifies the precision of the estimated LWL data at each time step of the time series.
• High-frequency variations: standard deviation of the high frequency signal within each time series (computed thanks to a Lanczos high-pass filter (Lanczos, 1988) with an arbitrary 1-month cut-off period). This indicator gives additional information on the lake water level precision for small lakes. For consistency reasons, it is estimated for lakes of all sizes. Indeed, it primarily quantifies remaining errors due to the geoid model, as well as the shifts in the satellite orbits and the inter-mission bias.
• Mean time step: Average time between two valid measures. Since the estimation of the lake water level is based on multiple missions with different repetition cycles and different ground tracks, the time step per lake is not regular. Moreover, measurements may also be missing due to the poor quality of data that has been automatically removed during the process. This indicator provides information on the average frequency of data available per lake.

These performance indicators were estimated for each lake for two time periods: the full time series of ~30 years for most lakes and the last 10 years. These indicators based on the last 10 years give the performance of recent quality of the products and provides insight into the future quality of the next version of the Thematic CDR (TCDR) and CDR LWL products.

Since LWL products are derived from multiple missions, other interesting indicators involve the comparison of the performance between missions. Missing values per mission are calculated for current missions: Jason-3, Sentinel-3A, Sentinel-3B and Sentinel-6A

1.3.2. Relative error assessment

External products using different data processing or acquisitions are useful to assess the quality of the LWL products. Two types of datasets are considered: data generated by altimetry products, and data obtained by in-situ measurements. These products use different datums, different dates, and, for the altimetry products, different altimetry missions or standards/tracks. Thus the comparison is not straightforward. However, it can provide information on the product's precision which are examined through this relative error assessment. It must however be thoroughly analysed to understand if the differences are within the products' uncertainties or errors in one of the two products.

For each lake, we estimate the difference of the variation and the Pearson coefficient to estimate the linear correlation between LWL time series.

2. Validation results

2.1. Absolute error assessment

The three performance indicators described above (see Section 1.3.1) were estimated for each lake (i.e., 229 lakes available in the most recent version 4) for two time periods: the full time series of ~30 years for most lakes, and the last 10 years. Annex A contains the values of those performance indicators for each lake for the two time periods. Performance indicators for the last 10 years are indicative of the recent and future quality. Figure 1 provides an overview of the performance indicators for all lakes and three lake size categories:

• Small lakes: with surface areas of less than 3000 km²
• Medium lakes: with surface areas between 3000 and 10000 km²
• Large lakes: with surface areas greater than 10000 km²

These general results are elaborated more thoroughly in the subsections below. However, they illustrate the general behaviour noted above, i.e., that the precision decreases with the size of the lakes. The dispersion decreases between the full period and the last 10 years only. This is most likely due to the improvement of the sensors, but also the increase on the precision of orbits measurements thanks to DORIS system (Doppler Orbitography and Radiopositioning by Satellite) and other geophysical corrections. Additionally, a higher number of samples, for the full period, could also have an impact in the dispersion estimation. However, the high-frequency variations increase because more satellites and more tracks are used in the products (decreasing the mean time step), which induces inter-calibration issues as well as uncertainties related to the limitations of the geoid models.

Figure 1: Performance indicators for the overall period (1992-2022. 139 lakes have a temporal coverage of more than 10 years) in dark colours compared to the last 10 years (2013-2022 period) in light colours for three categories of lakes depending on their size (arranged left to right per indicator group as small lakes: less than 3000km², medium lakes: between 3000 and 10000 km², and large lakes: larger than 10000 km²).

The overall dispersion has decreased compared to version 3 (based on a smaller number of lakes: 166 in version 3 vs 229 lakes in version 4) because the new lakes in version 4 are monitored by newer and better performing missions. However, the dispersion of the lakes can be very variable.. Depending on the location of the track over the lake, mainly near the shore, the backscatter signal is affected by land contamination as near water bodies or any reflecting surface. For some of them, outliers increase the median dispersion, as is the case of Lake Mangbeto (Togo) for example. As mentioned above, in most of the cases, the dispersion changes over time and decreases in recent missions, as in the case of Lake Hyargas (Mongolia), Lake Kairakum (Tadjikistan) or Lake Bangweulu (Zambia) (Figure 2).

Compared to version 3, the high frequency variation in the most recent version 4 has decreased because several new lakes in version 4 are crossed by a single mission, which avoids bias between tracks and missions. Finally, the median Time step increased, mainly in the small lakes, as some of them are only overpassed by the Sentinel-3A or Sentinel-3B mission with a revisiting time of 27 days.

(a)

(b)

(c)

(d)
Figure 2: Dispersion of the estimated LWL data over time for Lakes (a) Mangbeto in Togo, and (b) Hyargas in Mongolia (c) Kairakum in Tadjikistan, and (d) Bangweulu in Zambia.

2.1.1. Along-track dispersion

The median transect dispersion per lake is less than 10 cm for medium and large lakes, in line with the threshold in the product requirements. For small lakes, the median dispersion is approximately 11 cm for the overall period but decreases over the recent 10-year period. Several of these new small lakes are monitored by recent missions, such as Sentinel-3A and Sentinel-3B. These missions feature improved along-track resolution (approximately 300 m) in SAR mode which facilitates the measurement of small lakes. As indicated in the previous section, land contamination in the footprint is one of the main sources of error in altimetry over inland waters. There is a higher probability to have land contamination with small lakes, which increases the dispersion, whereas large lakes tend to provide similar results for altimetry to what is expected for oceanic surfaces.

Figure 3 shows the dispersion per lake with respect to the size of lakes, for those lakes which feature at least 20 years of data coverage. Lake Lagdo (Cameroon), a small lake of 586 km², has the greatest mean dispersion (33 cm) with a great variability over time, depending on the mission monitoring the lake (Figure 4). This lake was initially monitored by Envisat (2002-2012) and is currently being monitored by Sentinel-3A, starting in 2016. The Cryosat data have made it possible to fill some of the data gaps in this lake between 2012 and 2016.

Figure 3: Mean dispersion of the estimated LWL data in relation to lake surface (surface in logarithmic scale, x-axis), for those lakes with over 20 years of data coverage (117 lakes). Red lines indicate the thresholds bounding the lake size categories (ordered left to right as small lakes: less than 3000km², medium lakes: between 3000 and 10000 km², and large lakes: larger than 10000 km²).

Figure 4: Temporal development of dispersion of LWL estimates for Lake Lagdo (Cameroon), a small lake of 586 km² with the greatest mean dispersion (33 cm).

In addition to an individual analysis by lake over the entire period, it is also important to analyse the temporal development of dispersion over the past few years. This information is useful for assessing the quality of the most recent measurements, which is expected to be better, thanks to the improvement of the sensors and the ground segments. Figure 5 shows the mean dispersion for all lakes over the full period (1992-2022) and over the last 10 years (2013-2022) ordered by lake size. Only lakes with at least 20 years of temporal coverage are provided in Figure 5. As expected, the dispersion decreases for almost all of them, and is lower for larger lakes.

Figure 5: Dispersion per lake (y axis), arranged by lake size from smallest to largest (x axis, left to right), comparing dispersion over the full period (1992-2022) to that of the last 10 years (2013-2022) for lakes with at least 20 years of data coverage (117 lakes).

In general, the shape of the lake and the position of the ground tracks have a significant impact on the quality of the lake water level estimation. If we analyse the Lagoa dos Patos (Brazil), a lake with a surface area of 10000 km², just at the threshold between medium to large size lakes, the dispersion has changed considerably with the altimetric missions (Figure 6). During the period covered by TOPography EXperiment (TOPEX)/Poseidon (Figure 7a), only one near-land track is available. With Geosat-Follow-On (GFO) (launched in 1998), a second track, with a better location but also near land, crosses the lake (Figure 7b). Thanks to ENVISAT (launched in 2002), several well-positioned ground tracks became available (Figure 7c). Then in 2008, Jason-2 data improved the quality of the estimated LWL product along the same ground tracks as TOPEX/Poseidon. Finally, Sentinel-3A allowed an increase in quantity and quality of the lake water level estimation (Figure 7d) with several tracks over the lake and a globally improved system.

Figure 6: Change over time of the water level estimates (top) and dispersion (bottom) for the Lagoa dos Patos in Brazil (surface area of 10,000 km², a medium-large size lake). The dispersion has changed considerably with the advent of new altimetric missions to the measuring constellation (see also Figure 7). 

Figure 7: Ground Tracks over passing the Lagoa dos Patos in Brazil. TOPEX/Poseidon in red, GFO in green, ENVISAT in Yellow and Sentinel-3A in blue).

2.1.2. High-frequency variations

The second indicator concerns the high frequency signal variations (Figure 8). These variations are evaluated by taking the standard deviation of one-month high-pass filtered LWL time series. They mainly contain "noise" due to measurement uncertainty as well as the geophysical signal of the high-frequency water level variations. The variation over the recent 10-year period is higher than over the full period. This is mostly because there have been more satellites, thus more individual measurements (lower time step), over the last 10 years. More measurements with their specific precision and geoid errors yield an increase in high-frequency signal amplitude. Measurement uncertainty estimates using this indicator are less than 10 cm on average, which are also within accuracy requirements for the lake product (see Section 4). 

Figure 8: High frequency variation (y-axis) per lake (x-axis), sorted by lake size from smallest to largest area (left to right), during the whole period (1992-2022) compared to the last 10 years (2013-2022) for lakes with at least 20 years of temporal coverage (117 lakes).

One of the examples of increasing high frequency variation over 1992-2022 is Lake Kariba located along the border between the Zambia and the Zimbabwe. Since 2016, six ground tracks of Jason-3 and Sentinel-3A overpass the lake (Figure 9). Thanks to that, the median time step decreased from nine days to four days, although the time series during the last period is noisier (Figure 10). 

Figure 9: Ground Tracks over passing Lake Kariba between Zambia and Zimbawe (TOPEX and Jason in red, Sentinel-3A in blue) since 2016. 

Figure 10: Water level (top) and dispersion (bottom) time series for Lake Kariba located along the border between the Zambia and the Zimbabwe.

2.1.3. Time Resolution

The median time step between two valid water level measurements strongly depends on the tracks per mission overpassing Figure 11. contains the median time step per lake, sorted by size, for lakes with at least 20 years of temporal coverage. It clearly shows that some lakes are over passed by a single track of a mission, with a time step close to the satellite revisit time: 27 days for lakes with a track from Sentinel 3 mission or 10 days for lakes with a track from the Jason family missions.

Due to a greater number of satellite tracks sampling the lakes during the two-altimeter era, the frequency of measurements is much higher. Nevertheless, it concerns mainly large lakes, with more probability to be monitored by multiple missions and multiple tracks. The other factor impacting the median time step is the percentage of edited or missing measurements. With the improvement of the altimetry products (sensors, geophysical correction, ground segment, orbits, etc), less measurements are identified as outliers and edited. However, the increase in the number of altimetry missions may not have an impact on temporal resolution as some lakes are not monitored by the new missions. Nevertheless, these missions, such as Sentinel-3B on a new orbit (different from Sentinel-3A and Jason family orbits) allow monitoring new lakes. 

Figure 11: Median time step (y-axis) per lake (x-axis), sorted by size from smallest to largest area (left to right), during the full period (1992-2022) compared to the last 10 years (2013-2022) for the 117 lakes with at least 20 years of temporal coverage.

Another interesting indicator is the percentage of missing values. This value represents the number of lake water level estimates that could not be derived from their associated altimetry echo for different reasons: quality of the signal, shift of the ground trajectory, fast change in the level that activates the editing of the estimate. These percentage values were estimated for the current missions: Jason-3, Sentinel-3A and Sentinel-3B and Sentinel-6A, for all lakes (229 in version 4) ,and for the three categories of lake size as defined in section 2.1 (Table 3).

Table 3: Percentage of missing values depending on the mission and the size of the lake.

(*) Currently, only one large lake, Lake Bagre, in Burkina Faso is monitored using data from Sentinel-3B.

2.2. Relative error assessment

2.2.1. Altimetry products

In the context of the time series comparison, for the altimetry products, monthly time series of LWL products from external sources (G-REALM, DAHITI, see Table 2).

Table 4: Regions defined for the C3S LWL product.

The Pearson coefficient gives information about the correlation between time series, a value near to 1 indicates a very good correlation exists between two time-series. In following sections, we compare this coefficient between C3S time series and those of the two altimetry-based datasets: G-REALM and Dahiti.

For the subsequent sections, it is important to note that the number of lakes monitored by the different datasets (from C3S and external sources) is not exactly the same. As such, there are differences between the number of lakes monitored by C3S (229 in the Version 4.0 LWL dataset collection) in the different regions and the number of lakes compared to external datasets, which results in this analysis only being able to focus on common lakes.

2.2.1.1. Northern Europe

Figure 12 shows the 25 lakes currently being monitored by C3S in the North of Europe. These lakes are mainly located in Sweden, Finland and Northern Russia. Figure 13 contains the Pearson coefficients showing a good correlation between time series for the lakes and time series available in external data sources (G)REALM or Dahiti). 

Figure 12: Lakes in the C3S lakes dataset located in Northern Europe. 


Figure 13: Pearson correlation coefficients (y-axis) between C3S monthly time series of LWL compared to G-REALM (in red) and Dahiti (in blue) time series for lakes (x-axis) in the North European region.

Lake Vattern, the second largest lake in Sweden, is currently being monitored by Sentinel-3A and Sentinel-6A missions. Figure 14 shows the monthly variation from C3S and G-REALM time series as well as the difference between both. The G-REALM time series is much noisier than C3S time series, which explains the low value of the correlation between both time series (i.e., low Pearson correlation coefficient).

Figure 14: Time series of monthly water level variation for Lake Vattern in Sweden (top; red: C3S, blue: Dahiti), and the difference between the two time series (bottom).
Concerning Lake Saratovskoye, a Russian lake with an area of more than 100000 km², the Pearson correlation coefficient between C3S and the G-REALM time series is high (0.857) but much lower with the Dahiti time series (0.621). It is probably due to some outliers evident in the Dahiti time series (Figure 15).

Figure 15: Time series of monthly water level variation for Lake Sartovskoye, in Russia (top; red: C3S, blue: Dahiti), and the difference between the two time series (bottom).

2.2.1.2. Southern Europe

Six lakes are currently monitored in the south of Europe (Figure 16): Lake Bodensee (Germany), Lake Kremenchutska (Ukraine), Leman (Switzerland), Prespa (Turkey), Don (Russia) and Tsimlyanskoye (Russia). All these lakes show a very good Pearson correlation coefficient value, always above 0.9, both with respect to the time series of water level variation from Dahiti and for G-REALM (Figure 17). 

Figure 16: Lakes in the C3S lakes dataset located in Southern Europe. 

Figure 17: Pearson correlation coefficients (y-axis) between C3S monthly time series of LWL compared to G-REALM (in red) and Dahiti (in blue) time series for lakes (x-axis) in the Southern Europe region.

2.2.1.3. North Africa

In this region, 19 lakes are monitored (Figure 18). Note that this region encompasses areas typically thought of as being Central Africa. The correlation of C3S monthly data to G-REALM and Dahiti time series, presenting high values of Pearson correlation coefficients, is shown in Figure 19. The lowest value of the Pearson coefficient is for Lake Ziway, in Ethiopia: 0.88.

Figure 18: Lakes in the C3S lakes dataset located in the North of Africa.

Figure 19: Pearson correlation coefficients (y-axis) between C3S monthly time series of LWL compared to G-REALM (in red) and Dahiti (in blue) time series for lakes (x-axis) in the North African region.

2.2.1.4. Southern Africa

All the lakes located in the region of Southern Africa, 21 lakes shown in Figure 20, exhibit a very good correlation with the time series of G-REALM or Dahiti (Figure 21). The lowest value of Pearson coefficient is for Lake Kinkony in Madagascar: 0.933. 

Figure 20: Lakes in the C3S lakes dataset located in the South of Africa.

Figure 21: Pearson correlation coefficients (y-axis) between C3S monthly time series of LWL compared to G-REALM (in red) and Dahiti (in blue) time series in Southern Africa.

2.2.1.5. Northern America

This region contains 25 lakes located north of the 50^th parallel in the North American continent (Figure 22). The values of the Pearson correlation coefficients are shown in Figure 23. 

Figure 22: Lakes in the C3S lakes dataset located in northern North America. 

Figure 23: Pearson correlation coefficients (y-axis) between C3S monthly time series of LWL compared to G-REALM (in red) and Dahiti (in blue) time series in northern North American region.

Most of the Pearson coefficients are above 0.8, with some exceptions. For example, for Lake Winnipeg (Canada), the value of the Person correlation coefficient is 0.55 when compared to Dahiti data. The comparison between both time series (C3S and Dahiti) is shown in Figure 24. Nevertheless, the Pearson correlation coefficient for this lake when compared to in-situ measurements from the Water Office of Canada is 0.916, showing a very good correlation (Annex F).

Figure 24: Time series of monthly Water level variation for Lake Winnipeg in Canada (top; red: C3S, blue: Dahiti), and difference between the two time series (bottom).

The lowest Pearson correlation coefficient value with respect to G-REALM time series is the one for Lake Claire in Canada (0.685). The variation in the water level for the two time series and their difference are shown in Figure 25. The C3S time series exhibits a lower high frequency variation. Additionally, compared to in-situ data (Water Office of Canada), Lake Claire data from C3S dataset has a very good correlation (0.971).

Figure 25: Time series of monthly Water level variation for Lake Claire in Canada (top; red: C3S, blue: G-REALM), and difference between the two time series (bottom).

2.2.1.6. Southern North America

This region includes 20 lakes in the North American continent, south of the 50^th parallel, and a few lakes in Central America (Figure 26). The value of the Pearson correlation coefficient for data of all the common lakes in C3S with G-REALM and Dahiti is above 0.9, indicating a very good correlation between time series (Figure 27). 

Figure 26: Lakes in the C3S lakes dataset located in Central and North America south of the 50^th parallel. 

Figure 27: Pearson correlation coefficients (y-axis) between C3S monthly time series of LWL compared to G-REALM (in red) and Dahiti (in blue) time series in the southern North America region.

2.2.1.7. South America

A total of 10 lakes from the South American region out of the 18 lakes included in the C3S V 4.0 dataset (Figure 28) were compared to the G-REALM and Dahiti time series. All the Pearson correlation coefficients are above 0.8. The lowest one is 0.858, for Lake Ranco in Chile (Figure 29).

Figure 28: Lakes in the C3S lakes dataset located in South America. 

Figure 29: Pearson correlation coefficients (y-axis) between C3S monthly time series of LWL compared to G-REALM (in red) and Dahiti (in blue) time series in the South American region.

2.2.1.8. Northern Asia

Four lakes in the North of Asia (above 50° north) are included in the C3S dataset: Lake Azhibeksorkoli and Lake Tengiz in Kazakhstan, Lake Hovsgol and Lake Uvs in Mogolia (Figure 30). Figure 31 shows the Pearson correlation coefficients for those lakes.

Figure 30: Lakes in the C3S lakes dataset located in northern Asia. 


Figure 31: Pearson correlation coefficients (y-axis) between C3S monthly time series of LWL compared to G-REALM (in red) and Dahiti (in blue) time series in northern Asia.

Concerning Lake Hovsgol, the relative low values for the Pearson correlation coefficient are caused by different reasons. When C3S is compared to Dahiti, the Pearson coefficient is 0.754, and the two time series present a very similar pattern after 2015 (Figure 32). However, the amplitude of the variation is not similar (Figure 32). G-REALM, with a Pearson coefficient of 0.612, also presents differences at low water levels (Figure 33), where the altimetry-based water level estimation may be more challenging. For this lake, a comparison using in-situ data (if available) would be very useful.

Figure 32: Time series of monthly water level variation for Lake Hovsgol in Mongolia (top; red: C3S, blue: Dahiti), and the difference between the two time series (bottom). 

Figure 33: Time series of monthly water level variation for Lake Hovsgol in Mongolia (red: C3S, blue: G-REALM) (top); and difference between the two time series (bottom).

For Lake Uvs, there is a Pearson correlation coefficient of 0.645 when comparing C3S and the Dahiti time series. The main differences exist at the beginning of the time series (Figure 34). After 2013, both water level time series have a similar variation, and Pearson correlation coefficient increases to 0.959 for period 2013 – 2022. 

Figure 34: Time series of monthly water level variation for Lake Uvs in Mongolia (top; red: C3S, blue: Dahiti), and the difference between the two time series (bottom).

2.2.1.9. South-West Asia

This region includes lakes in Asia, south of 50° latitude and west of 85° longitude. In this region, 41 lakes are currently being monitored by C3S (Figure 35). Time series for 20 of those lakes are available on G-REALM and/or Dahiti datasets with a good correlation given by the Pearson correlation coefficient (Figure 36). Three of them have a Pearson correlation coefficient between C3S data and other external time series lower than 0.8: Lake Sevan in Armenia and Lake Van in Turkey (with Dahiti time series) and Lake Sasykkol in Kazakstan (with G-REALM time series).

Figure 35: Lakes in the C3S lakes dataset located in South-West Asian region.

Figure 36: Pearson correlation coefficients (y-axis) between C3S monthly time series of LWL compared to G-REALM (in red) and Dahiti (in blue) timeseries in the South-West Asian region.

There are two Pearson correlation coefficient values for Lake Srisalam in India because G-REALM provides one time series per track. Thus, for this lake, two timeseries are available: one using data from Sentinel-3A (track 679) and another one using data from Jason family missions (track 218). C3S datasets provides one timeseries per lake which can be mono- or multi-mission. For this specific lake, the C3S data are based on Sentinel-3B data (track 266). There is a good correlation between C3S and both G-REALM time series, with Pearson correlation coefficients greater than 0.8.

For Lake Van, the Pearson coefficient value is low (0.53). The comparison of the timeseries (Figure 37), shows three distinct sections: before 2012, where variations are noisier and the difference contains some outliers; A second section between 2012 and 2016, where data from Dahiti is no available; and the third section, after 2016, where the time series of the two datasets, C3S and Dahiti, are very similar. 

Figure 37: Time series of monthly water level variation for Lake Van in Turkey(top; red: C3S, blue: Dahiti), and the difference between the two time series (bottom).

Regarding Lake Sevan with a Pearson coefficient of 0.713, the Dahiti time series is very noisy before mid-2013. However, after this date both time series are very similar (Figure 38).

Figure 38: Time series of monthly water level variation for Lake Sevan in Armenia (top; red: C3S, blue: Dahiti), and the difference between the two time series (bottom).

For the third lake, Lake Chardarya, the water level variation from G-REALM is very low when compared to C3S time series (Figure 39).

Figure 39: Time series of monthly water level variation for Lake Chardarya in Kazakhstan (top; red: C3S, blue: G-REALM), and the difference between the two time series (bottom).

2.2.1.10. South-East Asia

This region includes 41 lakes in Asia south of 50° latitude (the 50^th parallel), and east of 85° longitude (Figure 40). For south- western side of this region, while most of the lakes have a Pearson correlation coefficient above 0.8 (Figure 41), three lakes have a Pearson coefficient lower than this value: Lake Namco (0.783), Lake Hongze (0.61) and Lake Zhelin (0.483) in China.

Figure 40: Lakes in the C3S lakes dataset located in the South-East Asia region.

Figure 41: Pearson correlation coefficients (y-axis) between C3S monthly time series of LWL compared to G-REALM (in red) and Dahiti (in blue) timeseries in the South-East Asian region.

The relatively low value of the Pearson correlation coefficient for lake Namco compared to Dahiti is a product of the difference in the estimated water level in 2022. Indeed, there is a jump in this year (Figure 42). In-situ data for this period, if available, would be useful to identify the real fluctuation in the lakes water level.

Figure 42: Time series of monthly water level variation for Lake Namco in China (top; red: C3S, blue: Dahiti), and the difference between the two time series (bottom).

Regarding Lake Hongze (China), the Pearson correlation coefficient of 0.61 between C3S and G-REALM time series is due to the fact that the G-REALM time series is noisier and contains some outliers (Figure 43).

Figure 43: Time series of monthly water level variation for Lake Hongze in China (top; red: C3S, blue: G-REALM), and the difference between the two time series (bottom).

Regarding Lake Zhelin (China), the low correlation value (Pearson correlation coefficient = 0.483) is caused by the outliers in the C3S time series (Figure 44). The maximum change value for this lake needs to be revised. This maximum change value can be interpreted as the maximum permissible partial derivative of the water level as a function of time.

Figure 44: Time series of monthly water level variation for Lake Zhelin in China (top; red: C3S, blue: G-REALM), and the difference between the two time series (bottom).

2.2.1.11. Oceania

Currently, two lakes are monitored in this region as part of the C3S Lakes service (Figure 45): Corangamite (in Australia) and Pukaki (in New Zealand). The time series for Lake Corangamite is also available in G-REALM dataset. The Pearson correlation coefficient between C3S and this external data source for is very close to one (0.901), indicating a very good correlation.

Figure 45: Lakes in the C3S lakes dataset located in the Oceania region.

2.2.2. In-situ products

The in-situ investigations compared LWL estimates available through the C3S service, to in-situ measurements of water level for water bodies monitored and measured by various agencies around the world. Here, they are presented in groups of water bodies, grouped by data provider.

2.2.2.1. Agencia Nacional de Aguas e Saneamiento Basico (ANA)

The National Water Agency of Brazil provides information on in-situ water level measurements for reservoirs⁹.  The Pearson correlation coefficients between C3S data and these measurements are shown in Table 5, and it is very high: greater than 0.9.

Table 5. C3S ANA Indicators. Pearson correlation coefficients between C3S LWL data and in-situ water level measurements in reservoirs monitored by the Water Agency of Brazil (Agencia Nacional de Aguas e Saneamiento Basico, ANA)⁹.

Annex B contains the figures corresponding to the time series of the monthly variation and the difference between C3S and in-situ measurements for each of the three reservoirs in Table 5.

2.2.2.2. Hidricos Argentina

The information concerning the historical variation on the Water Level for several in-situ stations in Argentina was obtained online from the “Base de Datos Hidrologica Integrada” (BDHI)¹⁰. Six lakes in this database are monitored by the C3S service. Pearson coefficients for these lakes are indicated in Table 6.

Table 6. Comparisons between C3S LWL data and in-situ water level measurements for Argentinian lakes, using data provided by Hidricos Argentina. Pearson correlation coefficients between C3S LWL data and in-situ water level measurements are shown for lakes available in the National Water Information System of Argentina (“Base de Datos Hidrologica Integrada, BDHI”)¹⁰.

The lake with the lowest Pearson correlation coefficient is Buenos Aires Lake, also known as General Carrera Lake. There is a larger difference between C3S and in-situ data time series before 2016, where bias is observed (Figure 46). Some outliers in the Hidricos Argentina dataset after 2016 also contribute to the lower correlation coefficient. 

Figure 46: Comparison between C3S LWL data and in-situ water level measurements for Lake Buenos Aires (General Carrea), using data provided by the National Water Information System of Argentina ("Base de Datos Hidrologica Integrada, BDHI")¹⁰. Monthly variations in the lake water level time series of both datasets are shown (top; red: C3S, blue: in-situ), along with the difference between the two time series (bottom).

Annex C contains the figures corresponding to the time series of the variation for each of the lakes in the Hidricos Argentina dataset.

2.2.2.3. U.S. Army Corps of Engineer

The third source of in-situ data is the U.S. Army Corps of Engineers (Table 2). The monthly data for the Great Lakes in the USA is available online up to 2021. Table 7 contains the indicators for the Great Lakes and the figures for each lake are provided in Annex D. The results show an excellent agreement between the present C3S product and this in-situ dataset, with correlations above 0.97, even though some outliers are detected in the C3S products (mainly in 2014 and 2015).

Table 7. Comparisons between C3S LWL data and in-situ water level measurements for the Great Lakes in North America, using data provided by the U.S. Army Corps of Engineers (see Table 2). Pearson correlation coefficients between C3S LWL data and in-situ water level measurements are shown.

2.2.2.4. U.S. Geological Survey

The U.S. Geological Survey (USGS) provides information on water resources data collected mainly in the U.S. The USGS investigates the occurrence, quantity, quality, distribution, and movement of surface and underground waters, and disseminates the data to the public, state and local governments, public and private utilities, and other Federal agencies involved with managing the U.S. water resources. The information concerning monthly lake water level measurements for four lakes was obtained on-line from the USGS database (see Table 2). The Pearson correlation coefficient values are indicated in Table 8.

Table 8: Comparisons between C3S LWL data and in-situ water level measurements for lakes whose measurements are made available by the U.S Geological Survey (USGS). Pearson correlation coefficients between C3S LWL data and in-situ water level measurements are shown (see Table 2).

The correlation between C3S LWL and USGS time series for lakes in Table 8 is very high, with Pearson correlation coefficients close to 1. Annex E includes the comparison of variation for all four lakes between C3S dataset and USGS dataset.

2.2.2.5. Water Office of Canada

In-situ data for Canadian lakes is freely available through the Water Office of Canada (Table 2). The time series of 16 lakes thar are also being monitored by the C3S project were compared to this data. The Pearson correlation coefficients indicate that the correlation between time series is higher than 0.8 for most of the lakes (Figure 47). 

Figure 47: Pearson correlation coefficients between LWL monthly time series from C3S and in-situ dataset time series available through the Water Office Service of Canada.

Concerning lake Des Bois (Woods), a bias was found between the first and second part of the time series (Figure 48), probably due to the gap filling using data from Envisat in the C3S time series (period 2002-2009). However, this needs to be investigated further. 

Figure 48: Comparison between C3S LWL data and in-situ water level measurements for Lake Des Bois, using data provided by the Water Office Service of Canada. Monthly variation (top; red: C3S, blue: in-situ), and the difference between the two time series (bottom).

2.2.2.6. Swiss Federal Office for the Environment (FOEN)

The Swiss Federal Office for the Environment (FOEN, see Table 2) implements environmental monitoring programs, and maintains various measurement networks. It operates and coordinates several water-related monitoring networks. Moreover, it monitors water level of rivers and lakes in Switzerland. Currently, two Switzerland lakes are monitored in the C3S Lakes programme: Lake Bodensee and Lake Leman. Table 9 contains the values of the Pearson correlation coefficients comparing the level variation time series from both lakes.

Table 9: Comparisons between C3S LWL data and in-situ water level measurements for lakes whose measurements are made available by the Swiss Federal Office for the Environment (FOEN, see Table 2). Pearson correlation coefficients between C3S LWL data and in-situ water level measurements are shown.

The correlations for both lakes are close to one, indicating a very good correlation level. The figures with the comparison of the C3S and in-situ data time series are shown in Annex G.

3. Application(s) specific assessments

Currently, no application(s) specific assessments have been undertaken for the C3S lake water level dataset V4.0.

4. Compliance with user requirements

The requirements for the C3S LWL are described in the 2023 Target Requirements and Gap Analysis document [D1].

Table 10: Compliance of the C3S LWL with user requirements.

References

Lanczos, Cornelius (1988). Applied analysis. New York: Dover Publications. pp. 219–221. ISBN 0-486-65656-X. OCLC 17650089.

Schwatke, C., Dettmering, D., Bosch, W., and Seitz, F. (2015) DAHITI – an innovative approach for estimating water level time series over inland waters using multi-mission satellite altimetry, Hydrol. Earth Syst. Sci., 19, 4345-4364, https://doi.org/10.5194/hess-19-4345-2015, 2015.

Annex A. Performance indicators

Table A1. Comparisons between C3S LWL and in-situ data provided by external sources (monthly variations and difference).

Annex B. Comparison to lake water level data from the National Water Agency of Brazil (ANA)

Figure B1: Comparison between C3S LWL data and in-situ water level measurements for Lake Balbina in Brazil. Shown are monthly variation in water levels for both time series (top; red: C3S, blue: in-situ), and the difference between the two time series (bottom).

Figure B2: Comparison between C3S LWL data and in-situ water level measurements for Lake Sobradino in Brazil. Shown are monthly variation in water levels for both time series (top; red: C3S, blue: in-situ), and the difference between the two time series (bottom).

Figure B3: Comparison between C3S LWL data and in-situ water level measurements for Lake Tres Marias in Brazil. Shown are monthly variation in water levels for both time series (top; red: C3S, blue: in-situ), and the difference between the two time series (bottom).

Annex C. Comparison to lake water level data from the National Water Agency of Argentina

Figure C1: Comparison between C3S LWL data and in-situ water level measurements for Lake Argentino in Argentina. Shown are monthly variation in water levels for both time series (top; red: C3S, blue: in-situ), and the difference between the two time series (bottom).

Figure C2: Comparison between C3S LWL data and in-situ water level measurements for LakeCochrane in Argentina. Shown are monthly variation in water levels for both time series (top; red: C3S, blue: in-situ), and the difference between the two time series (bottom).

Figure C3: Comparison between C3S LWL data and in-situ water level measurements for Lake Fontana in Argentina. Shown are monthly variation in water levels for both time series (top; red: C3S, blue: in-situ), and the difference between the two time series (bottom).

Figure C4: Comparison between C3S LWL data and in-situ water level measurements for Lake General Carrera in Argentina. Shown are monthly variation in water levels for both time series (top; red: C3S, blue: in-situ), and the difference between the two time series (bottom).

Figure C5: Comparison between C3S LWL data and in-situ water level measurements for Lake San Martin in Argentina. Shown are monthly variation in water levels for both time series (top; red: C3S, blue: in-situ), and the difference between the two time series (bottom).

Figure C6: Comparison between C3S LWL data and in-situ water level measurements for Lake Viedma in Argentina. Shown are monthly variation in water levels for both time series (top; red: C3S, blue: in-situ), and the difference between the two time series (bottom).

Annex D. Comparison to lake water level data from the U.S. Army Corps of Engineers

Figure D1: Comparison between C3S LWL data and in-situ water level measurements for Lake Erie in the USA. Shown are monthly variation in water levels for both time series (top; red: C3S, blue: in-situ), and the difference between the two time series (bottom).

Figure D2: Comparison between C3S LWL data and in-situ water level measurements for Lake Huron in the USA. Shown are monthly variation in water levels for both time series (top; red: C3S, blue: in-situ), and the difference between the two time series (bottom).

Figure D3: Comparison between C3S LWL data and in-situ water level measurements for Lake Michigan in the USA. Shown are monthly variation in water levels for both time series (top; red: C3S, blue: in-situ), and the difference between the two time series (bottom).

Figure D4: Comparison between C3S LWL data and in-situ water level measurements for Lake Ontario in the USA. Shown are monthly variation in water levels for both time series (top; red: C3S, blue: in-situ), and the difference between the two time series (bottom).

Figure D5: Comparison between C3S LWL data and in-situ water level measurements for Lake Superior in the USA and Canada. Shown are monthly variation in water levels for both time series (top; red: C3S, blue: in-situ), and the difference between the two time series (bottom).

Annex E . Comparison to lake water level data from the U.S. Geological Survey

Figure E1: Comparison between C3S LWL data and in-situ water level measurements for Lake des Bois in the USA and Canada. Shown are monthly variation in water levels for both time series (top; red: C3S, blue: in-situ), and the difference between the two time series (bottom).

Figure E2: Comparison between C3S LWL data and in-situ water level measurements for Lake Great Saltin the USA. Shown are monthly variation in water levels for both time series (top; red: C3S, blue: in-situ), and the difference between the two time series (bottom).

Figure E3: Comparison between C3S LWL data and in-situ water level measurements for Lake Michigan in the USA. Shown are monthly variation in water levels for both time series (top; red: C3S, blue: in-situ), and the difference between the two time series (bottom).

Figure E4: Comparison between C3S LWL data and in-situ water level measurements for Lake Walker in the USA. Shown are monthly variation in water levels for both time series (top; red: C3S, blue: in-situ), and the difference between the two time series (bottom).

Annex F. Comparison to lake water level data from the Water Office of Canada

 
Figure F1: Comparison between C3S LWL data and in-situ water level measurements for Lake Athabasta in Canada. Shown are monthly variation in water levels for both time series (top; red: C3S, blue: in-situ), and the difference between the two time series (bottom).

Figure F2: Comparison between C3S LWL data and in-situ water level measurements for Lake Aylmer in Canada. Shown are monthly variation in water levels for both time series (top; red: C3S, blue: in-situ), and the difference between the two time series (bottom).


Figure F3: Comparison between C3S LWL data and in-situ water level measurements for Lake Baker in Canada. Shown are monthly variation in water levels for both time series (top; red: C3S, blue: in-situ), and the difference between the two time series (bottom).

Figure F4: Comparison between C3S LWL data and in-situ water level measurements for Lake Cedar in Canada. Shown are monthly variation in water levels for both time series (top; red: C3S, blue: in-situ), and the difference between the two time series (bottom).

Figure F5: Comparison between C3S LWL data and in-situ water level measurements for Lake Claire in Canada. Shown are monthly variation in water levels for both time series (top; red: C3S, blue: in-situ), and the difference between the two time series (bottom).

Figure F6: Comparison between C3S LWL data and in-situ water level measurements for Lake des Bois in the USA and Canada. Shown are monthly variation in water levels for both time series (top; red: C3S, blue: in-situ), and the difference between the two time series (bottom).

Figure F7: Comparison between C3S LWL data and in-situ water level measurements for Lake Greateslave in Canada. Shown are monthly variation in water levels for both time series (top; red: C3S, blue: in-situ), and the difference between the two time series (bottom).

Figure F8: Comparison between C3S LWL data and in-situ water level measurements for Lake Erie in the USA and Canada. Shown are monthly variation in water levels for both time series (top; red: C3S, blue: in-situ), and the difference between the two time series (bottom).

Figure F9: Comparison between C3S LWL data and in-situ water level measurements for Lake Huron in the USA and Canada. Shown are monthly variation in water levels for both time series (top; red: C3S, blue: in-situ), and the difference between the two time series (bottom).

Figure F10: Comparison between C3S LWL data and in-situ water level measurements for Lake Manitoba in Canada. Shown are monthly variation in water levels for both time series (top; red: C3S, blue: in-situ), and the difference between the two time series (bottom).

Figure F11: Comparison between C3S LWL data and in-situ water level measurements for Lake Nipissing in Canada. Shown are monthly variation in water levels for both time series (top; red: C3S, blue: in-situ), and the difference between the two time series (bottom).

Figure F12: Comparison between C3S LWL data and in-situ water level measurements for Lake Ontario in the USA and Canada. Shown are monthly variation in water levels for both time series (top; red: C3S, blue: in-situ), and the difference between the two time series (bottom).

Figure F13: Comparison between C3S LWL data and in-situ water level measurements for Lake Superior in the USA and Canada. Shown are monthly variation in water levels for both time series (top; red: C3S, blue: in-situ), and the difference between the two time series (bottom).

Figure F14: Comparison between C3S LWL data and in-situ water level measurements for Lake Williston in Canada. Shown are monthly variation in water levels for both time series (top; red: C3S, blue: in-situ), and the difference between the two time series (bottom).

Figure F15: Comparison between C3S LWL data and in-situ water level measurements for Lake Winnipegosis in Canada. Shown are monthly variation in water levels for both time series (top; red: C3S, blue: in-situ), and the difference between the two time series (bottom).

Figure F16: Comparison between C3S LWL data and in-situ water level measurements for Lake Winnipeg in Canada. Shown are monthly variation in water levels for both time series (top; red: C3S, blue: in-situ), and the difference between the two time series (bottom).

Annex G. Comparison to lake water level data from the Swiss Federal Office for the Environment (FOEN)

Figure G1: Comparison between C3S LWL data and in-situ water level measurements for Lake Bodensee in Germany, Swiss and Austria. Shown are monthly variation in water levels for both time series (top; red: C3S, blue: in-situ), and the difference between the two time series (bottom).

Figure G2: Comparison between C3S LWL data and in-situ water level measurements for Lake Leman in Swiss. Shown are monthly variation in water levels for both time series (top; red: C3S, blue: in-situ), and the difference between the two time series (bottom).

Related articles

• Page:
  ERB TSI: Product User Guide and Specification (PUGS)
• Page:
  ERB TSI: Product Quality Assurance Document (PQAD)
• Page:
  ERB TSI: Product Quality Assessment Report (PQAR)
• Page:
  ERB TSI: System Quality Assurance Document (SQAD)
• Page:
  ERB TSI: Algorithm Theoretical Basis Document (ATBD)