Contributors: B. Calmettes (CLS), N. Taburet (CLS), L. Carrea (University of Reading), C.J. Merchant (University of Reading)
Issued by: CLS/B. Calmettes
Date: 14/12/2022
Ref: C3S2_312a_Lot4.WP1-PDDP-LK-v1_202206_LWL_PQAD-v4_i1.1
Official reference number service contract: 2021/C3S2_312a_Lot4_EODC/SC1
History of modifications
List of datasets covered by this document
Related documents
Acronyms
General definitions
Accuracy: Closeness between the measured value and the true quantity value.
Bias: Estimate of a systematic error.
Precision: Closeness between measured values obtained by replicate measurements on the same object under similar conditions.
Uncertainty: Representation of the statistical dispersion attributed to a measured value.
Scope of the document
This document is the Product Quality Assurance Document (PQAD). It describes the dataset and validation methods as well as the strategies used for validation and characterisation of the Copernicus Climate Change Service (C3S) Lake Water Level product. It is a self-contained document which gathers all validation methods and analyses conducted to assess the quality of the C3S Lake Water Level product.
Executive summary
The C3S Lake production system (C3S ECV LK) provides an operational service, generating lake surface water temperature and lake water level climate datasets for a wide variety of users within the climate change community. The present document covers the lake water level system.
The Product Quality Assurance Document includes the validation methods and strategies used for the validation and characterisation of the accuracy and stability of the Lake Water Level (LWL) product described in the Target Requirement and Gap Analysis Document [D5]. This document is applicable to the Climate Data Record version 4.0 produced in August 2022 (product version v4.0).
The Quality assurance analysis for the lake water level consists of two distinct parts: (i) the absolute error with the validation of the data, and (ii) the estimate of the relative error by the comparison of generated products with external data. Quantifying absolute error is performed by analysing the error generated by the instruments and processing over time. The C3S lake water level product is based on measurements from several altimetry missions, with technology that has evolved and improved in consecutive missions (going from standard altimeters as Low Resolution Mode onboard of Jason-3 to Synthetic Aperture Radar - SAR onboard of Sentinel-6A). Estimating relative error is achieved by comparing the generated products with external data from either (i) other altimetry-based products or (ii) products derived from in-situ measurements.
This document describes the methodology at the product version 4.0 of the C3S LWL product. The first section describes the characteristics of the C3S product while the second section describes the characteristics of the external dataset used for the comparison. The last section includes the methodology to be used for the absolute assessment and the relative assessment of the product.
1. Validated products
1.1. Product Specifications
Presently, this section relies on statements for the Lake Essential Climate Variable (ECV) from Global Climate Observing System (GCOS), published literature, experience from other Climate Data Records (CDR) projects, and requirements emerging from the definition of the service as described in the Target Requirement and Gap Analysis Document [D5]. The summary of the user requirements is indicated in Table 1.
Table 1: User Requirements for Lake Water Level
Content of the dataset | |
Content of the main file | The data file shall contain the following information on separate layers:
|
Spatial and temporal features | |
Spatial coverage | The product shall be distributed globally based on a harmonized identification of the products. The target area of the lakes must be at least 1 km x 1 km. The global coverage is estimated as the number of lakes monitored. |
Temporal coverage | Times series of 10 years minimum are required. However, all duration timeseries are included in the C3S dataset to make data for the recent missions available, as Sentinel-3A and Sentinel 3-B have less than 10 years temporal coverage (launched in 2016 and 2018 respectively). This enables the coverage of new lakes not covered by other missions. |
Temporal resolution | Following GCOS requirements, the temporal resolution is daily. However, the time resolution for LWL, based on observations from multiple satellites, is irregular and ranges from 10 to 35 days depending on the mission repeat cycle. For large lakes, monitored by multiple missions, the temporal resolution may be higher. |
Data uncertainties | |
Target | 3cm for large lakes, 10 cm for the remainder as required by GCOS |
Format requirements | |
Format | NetCDF, CF Convention |
A detailed description of the product generation is provided in the Algorithm Theoretical Basis Document (ATBD) [D2], with further information on the product given in the Product User Guide and Specification (PUGS) [D4].
1.2. Available products
The C3S lakes products comprise a long-term CDR. The time series has been computed to cover the period from late 1992 to June 2022. The altimetry data is provided by the missions:
- Topex/Poseidon from 1992 to 2002
- Jason-1 from 2001 to 2013
- Jason-2 from 2008 to 2015
- Jason-3 from 2016 to 2022
- Envisat/RA-2 from 2002 to 2012
- SARAL/Altika: from 2013 to Present
- Sentinel-3A/SRAL from 2016 to Present
- Sentinel-3B/SRAL from 2018 to Present
- Sentinel-6A from April 2022 to Present
This current document is applicable to the Quality Assessment activities performed on the dataset generated in August 2022 (CDR v4.0).
1.3. Parameters and units
The products in this C3S version are shared with the Theia-land program1 supported by CNES2. The research leading to the current version of the product has received funding from CNES, LEGOS3 and CLS as well as the Copernicus Climate Change program permitting an increase in the spatial coverage.
The Lake Water Level (LWL) product, in netCDF4 format, contains:
- The measure of the absolute height expressed in meters of the reflecting water surface beneath the satellite with respect to a vertical datum (EGM2008 or GGM02C depending on the lake).
- The associated uncertainty expressed in meters.
Additional information is also included in the output file, concerning both the lake (name, country, basin, latitude, longitude) and the processing (processing mode, processing level).
2. Description of validating datasets
External altimetry-based data and in-situ data provide an accurate external estimate of water level, though they are not necessarily co-located in time and space, and contain specific errors.
In the context of the quality assessment, the LWL products can be compared with external data sets. Uncertainty of the LWL products can be quantified through the computation of the root mean square error (rmse) and their correlation with external data as reference. Such results will be presented in the Product Quality Assessment Report [D3] associated with v4.0 products.
2.1. Altimetry based datasets
2.1.1. G-Realm
The G-Realm4 dataset is produced by The United States Department of Agriculture (USDA) in cooperation with the National Aeronautics and Space Administration and the University of Maryland. This database utilises radar altimetry from different satellites (Topex/Jason, Envisat) to produce lake water level over global inland water bodies.
The result of the process approach is a time series of height variation. The accuracy value of the altimetric elevations is variable. Typical rmse values (excluding winter ice-on periods) range from a few centimeters for the largest of lakes with open, rough (wind-driven) surfaces, to 15-30cm rmse for smaller lakes or those with calm, sheltered surfaces. The presence of ice in lakes can cause erroneous height measurements due to potential radar penetration (affecting the radar altimeter Range estimate, and the atmospheric Range correction deduced by the onboard microwave radiometer).
2.1.2. Dahiti
Dahiti5 is a global database developed by the Deutsches Geodätisches Forschungsinstitut der Technischen Universität München6 and provides water level time series from satellite altimetry from different satellites (such as Topex/Poseion, the Jason family or the Sentinel family) on lakes, reservoirs, rivers and wetlands.
This database currently contains water level series for 962 lakes. The processing for generating the Dahiti products, based on an extended outlier detection and Kalman filtering, is described in (Schwatke et al., 2015). The accuracy of the water level time series varies between few centimeters for large lakes and few decimeters for small rivers. The water level time series in Dahiti are freely available and can be downloaded after a registration process.
2.2. In-situ datasets
Different sources of in-situ data from different countries provide either the water level or the variation of the water level in lakes also available in the C3S dataset.
Table 2: In-situ datasets used for validation purposes, and website links where further information can be found on them7.
In-situ data | |
http:/hydrolare.net | |
https://www.lre.usace.army.mil/ | |
https://snih.hidricosargentina.gob.ar/ | |
https://waterdata.usgs.gov/ | |
https://wateroffice.ec.gc.ca/ | |
The Federal Office for the Environment provides hydrological data, and in particular the water levels of lakes in Switzerland. | |
https://www.gov.br/ana/en |
3. Description of product validation methodology
3.1. Overall procedure
The validation exercise consists of validating the local changes of water level as measured by altimetry. The uncertainties or errors in the products are two-fold: i) measurement errors related to instrument characteristics and (ii) processing errors.
The following description of errors is based on (Bercher, 2008).
3.1.1. Instrument characteristics
Water Surface Height estimates are obtained by measurements of the satellite radar echo return time. The instrument monitors the reflected radar signal over a recording window (covering approximately 60 meter in range). This window can be determined by the on-board tracking system based on the analysis of the last range measurements to approximate the position of the useful information in the incoming waveform – this is known as close loop mode. It can also be imposed by the use of a Digital Elevation Model (DEM) stored on board the satellite – this is known as open loop mode.
In close loop mode, in case of rapid and unexpected changes of the topography, the tracking system may not position the recording window accurately and fail to measure the range (“unhook”). In this case, the system misses between 1-3 seconds of measurements [Chelton et al., 2001]. This ultimately leads to the loss of 10 to 120 high frequency measurements, depending on the sampling rate of the altimeter, or 5 to 20 km. The open loop mode allows avoidance of such problems provided an accurate enough DEM is uploaded.
All current missions, Sentinel-3A, Sentinel-3B and Sentinel-6A operating a SAR, use these two modes. Switching between the modes is defined by the position of patches uploaded onboard. The first DEM uploaded to Jason-3 included about 250 lakes from the Hydroweb product. It has been regularly updated and is currently set to cover 20 000 lakes for Sentinel-6A (June 2020 version) and more than 40 000 lakes each for Sentinel-3A and Sentinel-3B (August 2021)
The present product contains both lakes for which open and close loop acquisition modes are used. An evaluation of the products has not been performed as a function of the acquisition mode, nevertheless the editing algorithm [ATBD, D2] ensures that the accepted Water Level values are compatible with the a priori elevation provided in the lake parameter files. Therefore, such a selection acts as a posteriori use of the DEM. It ensures that altimeter hooking is detected on possible different targets. Associated measurements can then be rejected by the water level processing chain.
Other parameters can also affect the water level retrieved by the altimeter. In the context of hydrology, the presence of land surfaces in the altimeter footprint may result in a significant error in the range estimate from which water level is computed. This effect is particularly significant when land surfaces contain echogenic targets (such as buildings, emerged lands, humid vegetation, dams etc.). The surface of the footprint, thus, plays an important role.
All altimeters are not similarly affected by surface specificities or canopy penetration of the signal. For example, it depends on the altimeter frequency band (e.g. C, Ka, Ku) and the technology of the altimeter. Two technologies are currently used. Products are derived from either standard altimetry measurements (low resolution mode, LRM, for Jason-3) or Synthetic Aperture Radar measurements (SAR, for Sentinel-3A, Sentinel-3B and Sentinel-6A). Lakes time series can contain measurements derived from both technologies, provided by Sentinel-3A/Sentinel-3B and Jason-3/Sentinel-6A.
3.1.2. Processing characteristics
Though altimeters are perfectly suited to the observation of sea level, their performance decreases over inland waters, and particularly small lakes. The smaller the ratio between water and emerged land surfaces in the footprint, the more complex the waveforms. This requires dedicated retracking algorithms which are not yet capable of reaching the quality of the retracking algorithm over ocean.
Instrumental, propagation and geophysical corrections are also used in the computation of water level and may contain errors. The most notable is the geoid correction, locally leading to uncertainties of up to 35 cm (Jekeli and Dumrongchai, 2003).
Consequently, several criteria are applied to remove segments of satellite passes (transects) which are not compliant with the quality requirements as described in the Algorithm Theoretical Basis Document [D2]. The validation and quality assessment metrics presented in this document only refer to these pre-selected transects.
3.2. Validation approach
The main problem of water level validation is that one tries to quantify errors smaller than natural variability.
Thus, one needs to process separately the errors arising from different sources (see Sections 3.1.1 and 3.1.2). Relevant questions include: i) are they constant in time (for a given lake, but variable on another target), or in space (for a given pass/day, all targets "see" the same error), or both (bias), or neither (random)? (ii) does the use of a given methodology (difference at crossovers, in-situ) to separate errors and/or natural variability achieve better results than performing an absolute validation (statistics)?
The validation methods presented below are, thus, based on both absolute and relative approaches.
The first absolute validation method quantifies the dispersion of consecutive water level high frequency measurements for the same target by computing their standard deviation. Note that consecutive measurements are then averaged to compute the lake products. The more precise the product is, the smaller the dispersion is expected to be, provided that one can accurately correct for the natural spatial slope over the transect.
Relative validation methods are based on a comparison of lake products with external water level data. These are either equivalent altimetry data or in-situ data.
3.3. Absolute Assessment
The validation exercise consists of validating the local changes of water level as measured by altimetry. The uncertainties or errors in the products are two-fold: measurement errors and processing errors. The following description of errors is based on (Bercher, 2008).
Four performance indicators are chosen to assess the quality of lake products:
- Dispersion: mean valid transect dispersion
- High-frequency variations: standard deviation of residuals from a high-pass Lanczos filter with an arbitrary 1-month cut-off period
- Mean time step: average time between two valid measures
- missing values: this is the percent of lake water level values that cannot be estimated due to different reasons (such as the quality of the signal, shift of the ground trajectory, fast change in the level that activates the editing of the estimate).
These performance indicators will be calculated for each lake (221 lakes in V4.0) for two time periods: i) the full time series of ~25 years for most lakes, and (ii) the last 10 years. The 10-year indicators give the performance of recent missions and a predictability of the quality of future products.
Lake products can contain altimeter data from multiple satellites tracks as well as different missions. Transects (intersections between satellite tracks and lakes) are on average longer in big lakes. So, the precision of the level estimation is also a function of the size of the lake.
Since LWL products are derived from multiple missions, other relevant indicators involve the comparison of the performance between missions. Missing values and the uncertainty per mission will be calculated, providing an indication of data quality improvements in recent missions.
3.4. Relative Assessment: comparison to independent altimetry products
External altimetry products, derived using different data processing, are useful when assessing the quality of the lake water level products. This data comes from two different products (G-REALM and Dahiti). These products use different datums and the comparison is not straightforward. The selected approach is the comparison of monthly anomalies per lake.
3.5. Relative assessment: comparison to in-situ data
In-situ data provide an accurate external estimate of water level, though it is not necessarily collocated in time and space. In addition, it also contains specific errors.
With this method, the lake water level products can be compared with in-situ data in terms of root mean square of the difference between time series and their correlation (Crétaux et al, 2016).
References
Bercher N. (2008). Précision de l'altimétrie satellitaire radar sur les cours d'eau: Développement d'une méthode standard de quantification de la qualité des produits alti-hydrologiques et applications (Doctoral dissertation, Doctorat AgroParisTech, Institut des Sciences et Industries du Vivant et de l'Environnement, spécialité Télédétection, AgroParisTech).
Biancamaria S., Schaedele, T., Blumstein D., Frappart F., Boy F., et al. Validation of Jason-3 tracking modes over French rivers. Remote Sensing of Environment, Elsevier, 2018, 209, pp.77-89. DOI: 10.1016/j.rse.2018.02.037
Chelton D. B., Esbensen S. K., Schlax M. G., Thum N., Freilich M. H., Wentz, F. J., ... & Schopf, P. S. (2001). Observations of coupling between surface wind stress and sea surface temperature in the eastern tropical Pacific. Journal of Climate, 14(7), 1479-1498.
Crétaux, J. F., Abarca-del-Río, R., Berge-Nguyen M., Arsen A., Drolon, V. Clos, G., & Maisongrande, P. (2016). Lake volume monitoring from space. Surveys in Geophysics, 37(2), 269-305.
Jekeli C., & Dumrongchai, P. (2003). On monitoring a vertical datum with satellite altimetry and water-level gauge data on large lakes. Journal of Geodesy, 77(7-8), 447-453.
Schwatke C., Dettmering D., Bosch, W., and Seitz F. (2015): DAHITI – an innovative approach for estimating water level time series over inland waters using multi-mission satellite altimetry, Hydrol. Earth Syst. Sci., 19, 4345-4364, DOI: 10.5194/hess-19-4345-2015
