Last modified on Sept 04, 2023 16:39

Contributors: L. Carrea (University of Reading), C.J. Merchant (University of Reading), B. Calmettes (CLS)

Issued by: University of Reading/L. Carrea, C.J. Merchant

Date: 07/10/2022

Ref: C3S2_312a_Lot4.WP1-PDDP-LK-v1_202206_LSWT_PQAD-v4_i1.1

Official reference number service contract: 2021/C3S2_312a_Lot4_EODC/SC1

History of modifications

List of datasets covered by this document

Related documents

Acronyms

General definitions

L2P – Geophysical variables derived from Level 1 source data on the Level 1 grid (typically the satellite swath projection). Ancillary data and metadata added following GHRSST Data Specification.

L3U – Level 3 Un-collated data are L2 data granules (cells) remapped to a regular latitude/longitude grid without combining observations from multiple source files. L3U files will typically be "sparse", corresponding to a single satellite orbit.

L3C – Level 3 Collated data are observations from a single instrument combined into a space-time grid. A typical L3C file may contain all the observations from a single instrument in a 24-hour period.

L3S – Level 3 Super-collated data are observations from more than one satellite that have been gridded together into a single grid-cell estimate, for those periods where more than one satellite data stream delivering the geophysical quantity has been available.

Brokered dataset – 'Dataset provided by another institution/initiative and not produced within this service.

Scope of the document

This document describes the strategies used for validation and characterization of the Lake Surface Water Temperature product (LSWT v4.5, brokered) produced under the European Space Agency’s (ESA) Climate Change Initiative (CCI) Lakes project [Carrea, Crétaux et al 2022], and extensions for LSWT generated as Climate Data Records (CDR) for the Copernicus Climate Change Service (C3S). The document includes also a description of the strategies used for quality assurance of the LSWT product prior to production.

Executive summary

The C3S Lake production system (C3S ECV LK) provides an operational service, generating lake surface water temperature and lake water level climate datasets for a wide variety of users within the climate change community. The present document covers the lake surface water temperature component.

The ESA CCI Lakes LSWT v4.5 data (brokered) [Carrea, Crétaux et al 2022] and their uncertainties were assessed under the CCI Lakes project using an in-situ reference dataset, which is collated annually at the end of each year. Robust statistics (median and robust standard deviation) were used when assessing differences between the reference data and the ESA CCI Lakes LSWT data, in order to obtain fair results that are not dominated by outliers and bad data (as defined in The Recommended GHRSST Data Specification, 2022) that arise in both the satellite-derived and validation datasets.

This Product Quality Assurance Document includes the definition and description of the datasets used (satellite and in-situ measurements), validation methods and strategies used for the validation based on satellite minus in-situ difference analysis and characterization of the accuracy of the Lake Surface Water Temperature product. The validation is carried out at the end of the L3S processing. Regarding the stability of LSWT on multi-annual scales, currently it is not possible to assess it due to the lack of reference data of quantified stability.

This document describes the methodology for the fourth version of the C3S LSWT products: contractual version 4 of the C3S Climate Data Record extension to be produced in January 2023. It contains:

1. A summary of the activities carried out for the ESA CCI Lakes LSWT-4.5 data (L3S series data). The planning of the validation and its execution was conducted as part of that project.
2. An overview of the activities carried out for C3S data production, which includes monitoring of statistics of the data used within the production scheme and validation against a reference dataset.

1. Validated products

1.1. Product Specifications

At the moment, this section relies on statements for the Lake Essential Climate Variable (ECV) from the Global Climate Observing System (GCOS), published literature, experience from other CDR projects, and requirements emerging from the definition of the service. The requirements will be updated in the future using requirements that emerge from users of the service and their feedback, and from any user requirements survey that has been undertaken in the ESA CCI Lakes project. The user requirements have been very recently updated and the latest GCOS requirements are indicated in Table 1.

Table 1: User Requirements for Lake Surface Water Temperature as described in GCOS (Updated version of the ECV requirements according to the GCOS Implementation Plan, not publicly available at the time of writing this document).

A detailed description of the product generation is provided in the Algorithm Theoretical Basis Document (ATBD) [D3], with further information on the product given in the Product User Guide and Specifications (PUGS) [D5].

1.2. Available products

This document describes the validation of the ESA CCI Lakes LSWT v4.5 product, including LSWTs and their evaluated uncertainty.
The LSWT v4.5 (scientific version number, noting it is different to the C3S product version which is C3S CDR version 4.0) C3S product provides a long-term climate data record (CDR) covering 1995 to 2022 (28 years). This combines the brokered data (see definition if the concept of brokered dataset is unclear) product which covers from 1995 up to 2020, and the extension CDR data which covers from 2021 until Sep 2022.The thermal satellite observations were provided by the following instruments:

1. ATSR2 on ERS-2 from 1995 to 2003
2. AATSR on Envisat from 2002 to 2012
3. MODIS on Terra from 2000 to 2022
4. AVHRR on MetOp-A from 2007 to 2019
5. AVHRR on MetOp-B from 2017 to Aug 2019
6. SLSTR on Sentinel-3A from Sep 2016 to 2022
7. SLSTR on Sentinel-3B from 2020 to 2022

This Product Quality Assurance Document is applicable to the Quality Assessment activities performed on the ESA CCI Lakes dataset dated in June 2022 (CCI_Lakes v2.0.2 dataset) and described in the Product Validation and Intercomparison Report available at the ESA CCI Lakes dataset. These activities for the C3S product will be reported in the Product Quality Assessment Report [D4].

1.3. Parameters and units

The LSWT product consists of global files in netCDF4 format, and it contains:

1. The best estimation of the Lake Surface Water Temperature (LSWT_skin) expressed in kelvin
2. The LSWT uncertainty associated with each lake pixel expressed in kelvin, which summarises the radiometric noise and the uncertainty in the retrieval as defined in Sec. 1 of the ATBD [D3].
3. The associated quality level which captures the confidence in the retrieval as defined in Sec. 4 of the ATBD [D3].

Additional information is also included in the output file, concerning the lakes identifiers, the instruments used for the observations, and if an inter-sensor adjustment has been applied.

2. Description of validating datasets

A match-up dataset was constructed from the in-situ temperature data collected through the ARCLake, the GloboLakes and the EU Surface Temperature for All Corners of Earth (EUSTACE), the C3S and the ESA CCI Lakes projects. Currently, this dataset consists of 160 observation locations covering 44 of the ESA CCI lakes. Details of the in-situ data with their sources are given in Table 2 which reports all locations for the ESA CCI lakes where there are matches. However, new in-situ observations will be collected before the release of the Product Quality Assessment Report [D4].

Table 2: List of the in-situ measurements sources for the ESA CCI lakes

The plot of the geographical distribution of the 160 sites over 44 of the 2024 ESA CCI lakes is shown in Figure 1.
As the in-situ data are from a variety of sources, with different formats, considerable effort has been put in to consolidate this data to a standard format for use in ESA CCI Lakes, and to apply a quality control procedure which was partly automated and partly by inspection. The quality control procedure was initiated within the ARCLake project, updated within GloboLakes and ESA CCI Lakes (see links and project descriptions at www.laketemp.net). Moreover, the data have a range of characteristics:

1. the measurements have been taken at different depths up to 1 m;
2. the temporal sampling ranges from 15 minutes to few times a year;
3. for some locations the measurements are averages while for others they have been taken at the reported time.

None of the in-situ measurements which have been collected are accompanied by an uncertainty estimation.
Thus, the reference data for satellite LSWT is in a relatively unsophisticated and un-coordinated state internationally. As far as we are aware, the ongoing efforts at collecting as much information for LSWT validation as possible within the C3S service will be internationally significant.

Figure 1: Geographical distribution of the in-situ measurements location for the ESA CCI Lakes.

3. Description of product validation methodology

3.1. Overall procedure

The validation exercise consists of using independent data (in-situ measurements) to validate the lake surface water temperature derived from satellite measurements. The independent data are described in Section 0 and consist of a collection of quality controlled in-situ measurements from different institutions. The validation is performed in two phases. First, in-situ measurements are matched with Level 2 (L2P) satellite-derived LSWT, where the coordinates are the satellite coordinates. A first check of the agreement between in-situ and per sensor satellite observations is performed. Then, the Level 3 (L3) cell corresponding to a L2 match is identified, and the final validation of the L3S product is performed through robust statistics (median and robust standard deviation) in order to prevent outliers to influence the results.

3.2. Generation of L2 matchup database

A per-sensor matchup is created. This contains spatially and temporally coincident satellite and in-situ data. It also provides the reference and time of the in-situ location, and the associated LSWTs, quality level and uncertainty from the L2 LSWT product. The satellite observation element of the matchup is created on the basis of the following criteria:

1. Spatially within 3 km from the location of the in-situ measurement and
2. Temporally within 3 hours for the in-situ measurements where the measurement time was available, otherwise the day was matched.

3.3. Validation of the L3S ESA CCI Lakes LSWT v4.5

The validation of the L3S ESA CCI Lakes LSWT v4.5 product is carried out on the L3 cell corresponding to the L2 pixel where a match has been found (Section 3.2). The differences between the satellite LSWT and reference in-situ data are analysed using both standard (mean and standard deviation) and robust statistics (median and robust standard deviation), the latter being statistics that are resistant to the presence of outliers in the distribution of differences (if any). Time series of the absolute temperatures together with their satellite minus in-situ difference are generated for each value of the LSWT quality levels. In addition, per-sensor box plots of the satellite minus in-situ differences are produced for each LSWT quality level.

Note that a satellite minus in-situ difference is expected due to the skin effect. Infrared radiometers are sensitive to radiation emitted between the air-surface interface and 20mm below the interface while the in-situ measurements considered here are taken at a distance up to 1m from the air-surface interface. During the night, the surface of the water is generally cooler than the subsurface by ~0.2 K [Saunders, 1967], [Embury et al, 2012]. However, during the day, if the wind speed is low enough, thermal stratification due to solar heating contributes a positive offset to the difference in temperature between the radiometric lake surface and the in-situ measurement depth (up to 1 m).

The positive thermal stratification would be expected to be in the range <<1 K for most observations and but occasionally of order a few kelvins. The degree of near-surface stratification to be expected in different lakes depends on fetch, weather conditions (radiative balance and wind speed), the depth of in-situ measurement, and any local vertical mixing perturbations introduced by the presence of the in-situ measurement system. The aggregate effect of these factors is not currently well quantified.

Overall, it is plausible that for day time LSWT observations the mean stratification effect is in the order of one or a few tenths, as has been determined over the oceans. In summary, a geophysical contribution to the satellite minus in-situ temperature difference has the expected skin effect of -0.2 K, but other positive geophysical offsets are similar in magnitude and are difficult to quantify precisely. In this context, a mean agreement of the physics-based retrievals and validation within +/-0.2 K is a convincing result. In terms of scatter, as well as the retrieval uncertainty and variability in the vertical stratification effects, the scatter includes in-situ uncertainty and horizontal variability. Again, quantitative understanding of the scatter from these effects is not yet mature, and for this reason full uncertainty budget validation remains a research aspiration.

3.4. Validation of the uncertainty

The LSWT uncertainty estimate has been validated comparing the difference satellite minus in-situ temperatures and the correspondent LSWT and in-situ uncertainties. The following quantity is calculated for each match:

where T indicates temperature, for LSWT and in-situ as indicated in the subscripts. 𝜎 _means the standard deviation from measurement uncertainty (for LSWT and in-situ) and from real differences because of point-to-pixel representativity effects.

The in-situ measurements uncertainty is not known for the data we have. We explore two assumptions: 𝜎_INSITU=0.2 K, a value based on deployment of similar measurement technologies to the ocean, and 𝜎_INSITU=0.5 K which would be at the upper end of our expectations for in-situ uncertainty. The representativity effect is presently unquantified and we set it to 0 K for the present; neglecting representativity has the tendency to make the LSWT uncertainty look underestimated.

Lakes_cci products, 𝜎²_{𝐿𝑆𝑊𝑇} is context sensitive and varies from match to match, which is why the validation approach involves the calculation of the above metric: the distribution of Δ should be a Gaussian distribution with mean equal to 0 and standard deviation equal to 1 when all standard deviations are well estimated and the retrieval is unbiased relative to the in-situ and any mean geophysical effect.

3.5. Assessment of stability

An assessment of stability (i.e. degree of change of the statistical distributions of error over time) is not implemented. Currently, there is a lack of reference data whose continuity and long-term stability are well understood.

4. Summary of validation results

This section provides some keys results of the LSWT product validation only for the ESA CCI Lakes part of the CDR. The detailed results for the complete CDR will be included in the Product Quality Assessment Report [D4], together with the validation of the LSWT uncertainty.

4.1. Generation of L2 matchup database and validation of the L2 ESA CCI Lakes LSWT v4.5

The matchup is carried out per sensor. The checks over the L2 LSWT data from MODIS are reported here as it is a new instrument used for the C3S product. Table 3 reports the robust statistics (median and robust standard deviation) and the more traditional sample statistics (mean and standard deviation) per quality level (QL) and per sensor for the matches across all the lakes.

The agreement varies according to the quality levels in a way that is expected. The best agreement is for quality levels 4 and 5, which indicate a higher degree of confidence can be held in the validity of the uncertainty estimate. The absolute value of the median/mean of the satellite minus in-situ difference becomes higher has the quality levels increases, as expected.

Table 3: Global validation statistics from comparing L2 ESA CCI Lakes LSWT form MODIS with in-situ measurements. QL indicates the quality level associated with the satellite-derived LSWT provided as part of the product package. RSD stands for robust standard deviation (proportional to the median of the absolute deviations from the data median) and SD for standard deviation.

Figure 2 shows plots of the MODIS observations of Lake Superior for site 01 in 2019 and 2020, and Lake Superior for site 02 in 2019, together with the climatology as a reference. The satellite in-situ measurement difference is also displayed where the green line displays the difference between in-situ and satellite measurements of LWST for all data categorised as quality levels 4 and 5 together.

Such plots are generated for all locations and years with in-situ matches as part of validation for product quality assurance.

Figure 2: Yearly plots for site 01 and 02 (their geographical position is reported in the plot) on lake Superior (USA) for the MODIS sensor. The golden line and the yellow shade represent the 1996-2016 climatology and its standard deviation, the coloured dots with the error bar represent the satellite LSWT, each of the colours indicate a quality level (5 red, 4 cyan, 3 purple, 2 green and 1 grey) and the error bar the LSWT uncertainty. The black line represents all the in-situ measurements while the white dots are the in-situ matches. The green line represents the satellite minus in-situ difference only for LSWT quality level 4 and 5.

4.2. Validation of the L3S brokered LSWT v4.0 product

Given the matchup database created at L2, the validation of the final LSWT ESA CCI Lakes product is carried out on the corresponding L3 cell.

Figure 3 shows a box plot of the L3S satellite in-situ difference, broken down per quality level. It shows a consistent good agreement for higher quality levels. Quality level 2 data are reported in the dataset but they are not recommended to be used. Quality level 3 may be useable with care (specific inspection by users), while the LSWT with quality levels 4 and 5 are data deemed as being acceptable for use by the production team.

Figure 3. Box plot of the satellite in-situ difference per quality level.

Figure 4. Yearly plot of L3S LSWT from MODIS (on right) for one site over lake Erken in Sweden (marked in red in the left panel) for year 2014. The golden line and the yellow shade represent the 1996-2016 climatology and its standard deviation, the coloured dots with the error bar represent the satellite LSWT, each of the colours indicate a quality level (5 red, 4 cyan, 3 purple, 2 green and 1 grey) and the error bar the LSWT uncertainty. The black line represents all the in-situ measurements while the white dots are the in-situ matches. The green line represents the satellite minus in-situ difference only for LSWT quality level 4 and 5.

Figure 4 (on the left hand side) shows the location of the in-situ measurement site on lake Erken in Sweden, one of the smallest lakes in the ESA CCI Lakes selection. The geographical position is shown on the plot on left hand side where the blue dots represent the centre of the cell in a 1/120 deg grid of the Globolakes/CCI_Lakes mask [Carrea et al, 2015]/[Carrea, Merchant et al, 2022] for lake Erken. The plot of the L3S LSWT observations according to the quality levels together with the in-situ measurements is shown on the right-hand side of Figure 4. Their difference is also reported together with the climatology as a reference. This plot shows a striking agreement between satellite and in-situ value of the water temperature especially for high quality levels given that the lake is one of the smallest lakes to be observed with satellites of ~1km resolution.

5. Summary of quality assurance prior to data release (C3S extension of LSWT v4.5)

The quality checks are performed after the LSWT product is generated. They are performed to verify that the processing has not produced unreasonable results. This is a preliminary check and consists of the following steps:

• Time series of observations minus the climatology statistics where the number of observations per day globally is plotted, together with the mean and standard deviation of the difference. Thresholds are set to select cases to be manually examined.
• Checks on the max, min of the LSWT and its uncertainty are performed, using thresholds dependent on latitude.
• Spatial plots of 5 lakes at different latitudes and of different sizes are manually examined.

Newly generated data that have validation properties similar to those found (and illustrated in Section 4) for the ESA CCI Lakes LSWT v4.5 are deemed to have satisfied the quality assurance and will be reported on in the Product Quality Assurance Report (PQAR) [D4].

References

The Group for High Resolution Sea Surface Temperature Science Team, Piollé, Jean-François, Armstrong, Ed, Casey, Kenneth, & Donlon, Craig. (2022). The Recommended GHRSST Data Specification (GDS) (GDS 2.1 revision 0). Zenodo. https://doi.org/10.5281/zenodo.6984989

Carrea, L., Embury, O. and Merchant, C. J. (2015) Datasets related to in-land water for limnology and remote sensing applications: distance-to-land, distance-to-water, water-body identifiers and lake-centre co-ordinates. Geoscience Data Journal, 2(2). pp. 83-97. ISSN 2049-6060 doi:10.1002/gdj3.32

Embury, O., Merchant, C. J. and Corlett G.K. (2012) A reprocessing for climate of sea surface temperature from the along-track scanning radiometers: Initial validation, accounting for skin and diurnal variability effects. Remote Sensing of Environment, 116. pp. 62-78. ISSN 0034-4257 doi:10.1016/j.rse.2011.02.028

Saunders, P.M. (1967) The temperature at the ocean-air interface. Journal of the Atmospheric Science, 24. pp. 269-273. doi:0.1175/1520-0469(1967)024<0269:TTATOA>2.0.CO;2

Carrea, L., Merchant, C.j.,  Simis S. (2022). Lake mask and distance to land dataset of 2024 lakes for the European Space Agency Climate Change Initiative Lakes v2 (Version 2.0.1) [Data set]. Zenodo. https://doi.org/10.5281/zenodo.6699376

Carrea, L., Crétaux, J.-F., Liu, X., Wu, Y., Bergé-Nguyen, M., Calmettes, B., Duguay, C., Jiang, D., Merchant, C.J., Mueller, D., Selmes, N., Simis, S., Spyrakos, E., Stelzer, K., Warren, M., Yesou, H., Zhang, D. (2022): ESA Lakes Climate Change Initiative (Lakes_cci): Lake products, Version 2.0.2. NERC EDS Centre for Environmental Data Analysis, 06 July 2022.  doi:10.5285/a07deacaffb8453e93d57ee214676304.

Related articles

• Page:
  ECV - Glaciers, Ice Sheets, Lakes, Soil Moisture
• Page:
  IV Version 1.2; SEC Version 2.0; GMB Version 2.0: Product User Guide Specification (PUGS)
• Page:
  IV Version 1.2; SEC Version 2.0; GMB Version 2.0: System Quality Assurance Document (SQAD)
• Page:
  IV Version 1.2; SEC Version 2.0; GMB Version 2.0: Product Quality Assurance Document (PQAD)
• Page:
  IV Version 1.2; SEC Version 2.0; GMB Version 2.0: Product Quality Assessment Report (PQAR)