Contributors: T. Usedly (Deutscher Wetterdienst), G.E. Thomas (UKRI-STFC RAL Space)
Issued by: Deutscher Wetterdienst / Tim Usedly
Date:
Ref: C3S2_D312a_Lot1.2.1.7_202408_PQAR_ECV_CLD_SLSTR_v1.2
Official reference number service contract: 2021/C3S2_312a_Lot1_DWD/SC1
History of modifications
List of datasets covered by this document
Related documents
Acronyms
General definitions
List of Figures
List of Tables
Scope of the document
This document provides a description of the product validation results for the Sea and Land Surface Temperature Radiometer (SLSTR) v4.0 based Interim Climate Data Record (ICDR) of the Essential Climate Variable (ECV) Cloud Properties (CLD).
The dataset produced by RAL Space and Brockmann Consult (BC) under the Copernicus Climate Change Service (C3S) programme ranges from 01/2017 – 12/2023 and provides an Interim Climate Data Record (ICDR) to the brokered Thematic Climate Data Record (TCDR) from European Space Agency Cloud Climate Change Initiative (ESA’s Cloud_cci).
The TCDR is a brokered product based on processing of the (Advanced) Along-Track Scanning Radiometer ((A)TSR) onboard ERS-2 and Envisat by RAL Space for the ESA Cloud_cci programme and ranges from 06/1995 – 04/2012. Detailed validation methodology and results are presented in the Cloud_cci Product Validation and Intercomparison Report [D1].
The ICDR is derived with a five-year gap from SLSTR onboard the Sentinel-3A and -3B satellites spanning from 01/2017 – 12/2023
Executive summary
The Sea and Land Surface Temperature Radiometer onboard Sentinel-3A has provided data since January 2017. The launch of Sentinel-3B in October 2018 makes it possible to deliver not only individual data from both satellites but also a merged Sentinel-3A/3B product. The merged version (10/2018 - 12/2023) is validated against the following satellite-based datasets: MODIS, CALIPSO-CALIOP and CLARA-A3, as well as ECMWF’s Reanalysis product ERA5. In addition to the merged SLSTR version, a second version on a different grid (equal area in addition to equal angle) is provided for the period from 07/2022 to 12/2023 and also validated against the same reference datasets as the equal angle version of SLSTR.
Validation to these SLSTR derived products is described in the following chapters of this document: Chapter 1 provides a summary of the product validation methodology while chapter 2 presents the validation results. A detailed validation methodology can be found in the Product Quality Assurance Document (PQAD) [D2]. Chapters 3 and 4 discuss possible application specific assessments and compliances with user requirements, respectively. SLSTR shows good agreement to CALIPSO and ERA5 for Cloud Fraction (CFC), while larger differences occur for CLARA-A3 and MODIS. Significant differences occur for Cloud Top Height (CTH), Cloud Top Pressure (CTP) (except for MODIS) and Cloud Top Temperature (CTT). Also, Ice Water Path (IWP) and Liquid Water Path (LWP) show consistently higher values for the SLSTR dataset. Differences between the two provided grid versions from SLSTR are negligible and meet the goal requirement by GCOS.
Calculated biases are evaluated against the GCOS requirements and are summarized in Table 1.
1. Product validation methodology
Detailed information about the validation methodology can be found in the corresponding PQAD [D2], section 3. The validation process is separated into three parts: Data preparation (section 1.1), validation (section 1.2) and evaluation (1.3).
1.1 Data preparation
Table 1-1 provides a summary of the datasets used for the validation and their temporal availability, spatial- and temporal resolution.
Table 1-1: Following datasets and references are used for the validation
Dataset | Time | Spatial resolution | Temporal resolution | Variables used for validation |
SLSTR onboard Sentinel-3A1 | 01/2017 – 06/2022 | Monthly mean | 0.5°x0.5° |
|
SLSTR onboard Sentinel-3B1 | 10/2018 – 06/2022 | Monthly mean | 0.5°x0.5° |
|
Merged SLSTR product | 10/2018 – 12/2023 | Monthly mean | 0.5°x0.5° |
|
Merged SLSTR product on equal area grid | 07/2022 – 12/2023 | Monthly mean | 0.5°x0.5° |
|
CLARA-A3 | 10/2018 – 12/2023 | Monthly mean | 0.25°x0.25° | CFC, CTP, CTT, CTH, IWP, LWP |
MODIS | 10/2018 – 12/2023 | Monthly mean | 1°x1° | CFC, CTP, IWP, LWP |
ERA5 | 10/2018 – 12/2023 | Monthly mean | 0.25°x0.25° | CFC, IWP, LWP |
CALIPSO | 10/2018 – 06/20232 | Monthly mean | 1°x1° | CFC, CTP, CTH |
All datasets are, if necessary, remapped to 1°x1° spatial resolution by bilinear interpolation.
1.2 Validation
Following uncertainty metrics are calculated: Bias, Mean Bias and Mean Absolute Bias.
Bias is the difference of dataset and reference dataset for each month and grid box:
\( B_{i,j}=F_{Data,i,j}-F_{Ref,i,j} \ (1) \)With B as Bias and F as dataset/reference and i, j as indices. Prior to the bias calculation, the datasets are collocated and only grid point considered, where two (or more) datasets have valid values (not nan). Grid points with identical grid points set to nan for a different dataset are set to nan.
Mean Bias (MB) describes the overall bias with respect to a reference dataset. It is defined as the bias of two gridded data records and a subsequently calculation of the global spatial average. This results in one value per month which can be averaged over the whole time period.
\( MB=\frac{1}{m*n}*\sum_{i=1}^m\sum_{j=1}^n (w_j(B_{i,j})) \ (2) \)with MB as Mean Bias, i and j (m and n, respectively) as indices, w as cosine weighting factor and B as Bias.
Mean Absolute Bias (MAB) is a bias corrected uncertainty metric and calculated by subtracting the previously calculated MB from every grid box bias. Subsequently the same steps as for the calculation of the mean bias are applied.
\( MAB=\frac{1}{m*n}*\sum_{i=1}^m\sum_{j=1}^n w_j*|B_{i,j}-MB| \ (3) \)1.3 Evaluation
The previously calculated Mean Absolute Bias is used as evaluation against the requirements defined by the Global Climate Observing System (GCOS) in The 2022 GCOS ECVs Requirements (GCOS 245) [D3]. They are summarized in Table 1-2.
Table 1-2: Summary of requirements for cloud properties based on GCOS [D3]
Products | Requirement | CFC | CTT | CTH | IWP | LWP |
Horizontal Resolution | Goal (G) | 25 km | 25 km | 25 km | 25 km | 25 km |
Breakthrough (B) | 100 km | 100 km | 100 km | 100 km | 100 km | |
Threshold (T) | 500 km | 500 km | 500 km | 500 km | 500 km | |
|
|
|
| |||
Temporal Resolution | Goal (G) | 1 h | 1 h | 1 h | 1 h | 1 h |
Breakthrough (B) | 24 h | 24 h | 24 h | 24 h | 24 h | |
Threshold (T) | 720 h | 720 h | 720 h | 720 h | 720 h | |
|
|
|
| |||
Accuracy | Goal (G) | 3 % | 2 K | 0.3 km | 0.05 kg/m² | 0.05 kg/m² |
Breakthrough (B) | 6 % | 4 K | 0.6 km | 0.1 kg/m² | 0.1 kg/m² | |
Threshold (T) | 12 % | 8 K | 1.2 km | 0.2 kg/m² | 0.2 kg/m² | |
2. Validation results
Sections 2.1 – 2.6 show the validation results for the six variables CFC, CTP, CTT, CTH, IWP and LWP with a climatology of collocated, deseasonalized, centered and weighted global averages. After the collocation the seasonality of each dataset is removed from the climatology as well as the average of each dataset subtracted. Absolute values show larger differences depending on the variable and the centered climatologies allow to evaluate the stability.
2.1 Cloud Fractional Cover
Figure 2-1: Climatology of collocated, deseasonalized, centered and latitude-weighted global monthly means of Cloud Fractional Cover for SLSTR and reference datasets
The merged SLSTR product shows overall good agreement with the reference data records in terms of stability (see Figure 2-1) and representation of the annual cycle. Although it has the lowest average of cloud fraction (61%, compared to references with 62%-68%, see Figure 2-2) it shows best agreement to CALIPSO and ERA5 leading to fulfill the goal requirement defined by GCOS. Bias between the two different SLSTR grid versions are negligible (61% cloud fraction on average). Largest differences occur for MODIS with generally higher values on large parts of the globe (see Figure 2-3). Differences in CLARA-A3 are separated over land and ocean areas: CLARA-A3 appears to have higher values over the oceans whereas large parts of land (North/South America, Greenland, Europe, Asia) have a positive bias. Differences compared to ERA5 are biggest over the Antarctica and north of 60°N. While these biases are negative, 60°N-60°S shows a slight positive bias. Comparison with CALIPSO shows a similar pattern with negative biases from 60° poleward. Other than CLARA-A3, differences over the oceans are positive, while land areas show a negative anomaly.
Figure 2-2: Climatology of collocated and latitude-weighted global monthly means of Cloud Fractional Cover for SLSTR and reference datasets
Figure 2-3: Mean bias for Cloud Fractional Cover of SLSTR and reference dataset for the period 10/2018 (07/2022 for SLSTR on equal area grid) – 12/2023 (06/2023 for CALIPSO). References: SLSTR equal area grid (top left), CLARA-A3 (top right), MODIS (middle left), ERA5 (middle right) and CALIPSO (bottom left)
2.2 Cloud Top Pressure
Figure 2-4: Climatology of collocated, deseasonalized, centered and latitude-weighted global monthly means of Cloud Top Pressure for SLSTR and reference datasets
Cloud Top Pressure (as well as Cloud Top Height (see section 2.4) and Cloud Top Temperature (see section 2.3)) shows significant differences in comparison with CALIPSO and CLARA-A3 (Figure 2-6). The SLSTR versions on equal angle and equal area grid (653.17 hPa and 655.63 hPa) have a comparable average to MODIS (667.88 hPa) but much higher values compared to CALIPSO (439.80 hPa) and CLARA-A3 (502.88 hPa) (Figure 2-5). Figure 2-4 shows that the stability is overall good compared to MODIS and CALIPSO. CLARA-A3 shows a significant negative trend from early 2023 on which is due to increased satellite drift.
Figure 2-5: Climatology of collocated and latitude-weighted global monthly means of Cloud Top Pressure for SLSTR and reference datasets
Figure 2-6: Mean bias for Cloud Top Pressure of SLSTR and reference dataset for the period 10/2018 (07/2022 for SLSTR equal area grid) – 12/2023 (06/2023 for CALIPSO). References: SLSTR equal area grid (top left), CLARA-A3 (top right), MODIS (bottom left) and CALIPSO (bottom right)
CALIPSO as well as CLARA have positive biases all over the globe except for the eastern half of the Antarctica. Biases are highest in tropical regions indicating that SLSTR underrepresents high clouds (cirrus) which results in higher cloud top pressures.
2.3 Cloud Top Temperature
Figure 2-7: Climatology of collocated, deseasonalized, centered and latitude-weighted global monthly means of Cloud Top Temperature for SLSTR and reference datasets
Results for Cloud Top Temperature are similar to Cloud Top Pressure, with significant higher values for SLSTR (Figure 2-7, Figure 2-9). This is in accordance with Cloud Top Pressure/Height. CLARA-A3 should be treated with caution as a reference due to drifting afternoon orbits (NOAA-18/19) (Figure 2-7). Differences occur primarily in the topics (Figure 2-8). There are no differences between the equal area and equal angle grid version.
Figure 2-8: Mean bias for Cloud Top Temperature of SLSTR and reference dataset for the period 10/2018 (07/2022 for SLSTR equal area grid) – 12/2023. References: SLSTR equal area grid (left) and CLARA-A3 (right)
Figure 2-9: Climatology of collocated and latitude-weighted global monthly means of Cloud Top Temperature for SLSTR and reference datasets
2.4 Cloud Top Height
Figure 2-10: Climatology of collocated, deseasonalized, centered and latitude-weighted global monthly means of Cloud Top Height for SLSTR and reference datasets
Cloud Top Height is significantly higher for CALIPSO (8.03 km) and CLARA-A3 (6.71 km) (Figure 2-11) confirming the results for CTP and CTH. Differences between the two SLSTR grid versions are small (4.24 km and 4.21 km for equal angle and equal area, respectively). Figure 2-10 shows an overall good stability between SLSTR and CALIPSO and a similar negative trend as CLARA-A3 shows. In addition, a slight positive trend from early 2023 on is seen for all datasets. Figure 2-12 confirms the findings with negative anomalies in tropical regions and positive anomalies in over the Antarctica.
Figure 2-11: Climatology of collocated and latitude-weighted global monthly means of Cloud Top Height for SLSTR and reference datasets
Figure 2-12: Mean bias for Cloud Top Height of SLSTR and reference dataset for the period 10/2018 (07/2022 for SLSTR equal area grid) – 12/2023 (06/2023 for CALIPSO). References: SLSTR equal area grid (top left), CLARA-A3 (top right) and CALIPSO (bottom left)
2.5 Ice Water Path
Figure 2-13: Climatology of collocated, deseasonalized, centered and latitude-weighted global monthly means of Ice Water Path for SLSTR and reference datasets
Comparison of Ice Water Path and Liquid Water Path is limited to 50°N-50°S. While reference datasets MODIS and ERA5 show relatively good agreement to each other (also in terms of stability, see Figure 2-13). SLSTR has not just higher values (0.19 kg/m², compared to 0.08 kg/m² for CLARA-A3, 0.02 kg/m² for ERA5 and 0.05 kg/m² for MODIS) (Figure 2-14) but also a positive trend for the entire period of 10/2018 – 12/2023. Positive biases are seen all over the globe with highest values around the Inter Tropical Convergence Zone (Figure 2-15).
Figure 2-14: Climatology of collocated and latitude-weighted global monthly means of Ice Water Path for SLSTR and reference datasets
Figure 2-15: Mean bias for Ice Water Path of SLSTR and reference dataset for the period 10/2018 (07/2022 for SLSTR equal area grid) – 12/2023. References: SLSTR equal area grid (top left), CLARA-A3 (top right), MODIS (bottom left) and ERA5 (bottom right)
2.6 Liquid Water Path
Figure 2-16: Climatology of collocated, deseasonalized, centered and latitude-weighted global monthly means of Liquid Water Path for SLSTR and reference datasets
Validation of Liquid Water Path shows in general similar patterns compared to Ice Water Path: Reference datasets CLARA-A3 (average 0.08 kg/m²), ERA5 (0.06 kg/m²) and MODIS (0.03 kg/m²) (Figure 2-17) show good agreement to each other, while SLSTR (0.12 kg/m²) has higher values with a stronger annual cycle and trend (Figure 2-16). Positive bias is clearest visible over ocean areas, while there is no or partly a small negative bias over land area (e.g. parts of Africa and North America in comparison with CLARA-A3). Compared to CLARA-A3, one can see a negative bias in mountainous regions for IWP and LWP (e.g. Rocky Mountains, Andes, Alps and parts of the Himalayan region) (Figure 2-18).
Figure 2-17: Climatology of collocated and latitude-weighted global monthly means of Liquid Water Path for SLSTR and reference datasets
Figure 2-18: Mean bias for Liquid Water Path of SLSTR and reference dataset for the period 10/2018 (07/2022 for SLSTR equal area grid) – 12/2023. References: SLSTR equal area grid (top left), CLARA-A3 (top right), MODIS (bottom left) and ERA5 (bottom right)
3. Application(s) specific assessments
This section is not applicable. There are no additional application specific assessments known since the dataset has just been published.
4. Compliance with user requirements
The GCOS requirements [D3] for the ECV Cloud Properties are used to evaluate the compliance for different users needs. Tables 4-1 and 4-2 show the requirements as well as the results.
GCOS defines three requirements depending on user’s needs:
- Goal (G): The strictest requirement, indicating no further improvements necessary
- Breakthrough (B): Intermediate level between threshold and goal. Breakthrough indicates that it is recommended for certain climate monitoring activities
- Threshold (T): Minimum requirement
The SLSTR ICDR meets the breakthrough/target requirement for the horizontal/temporal resolution, respectively.
The accuracy for CFC (depending on the reference between -1.47% and -6.82%) meets the threshold requirement and partly the goal and breakthrough requirement (Table 4-1).
However, there are larger differences for CTH, CTP and CTT which do not meet the requirement (Table 4-2). Despite having higher values, SLSTR LWP and IWP meet the breakthrough (LWP) and threshold (IWP) requirements (Table 4-3).
It is worth mentioning, that the GCOS requirements, defined by the World Meteorological Organisation (WMO), are not focused on satellite-based data records but also on climate models. Satellite-based data records, especially historical observing systems, are often not able to achieve the requirements.
Table 4-1: Results of evaluation against GCOS requirements for SLSTR CFC
Products | Requirement | Values | Cloud Fractional Cover |
Horizontal Resolution | Goal (G) | 25 km |
Roughly 55 km at the equator
|
Breakthrough (B) | 100 km | ||
Threshold (T) | 500 km | ||
| |||
Temporal Resolution | Goal (G) | 1 h |
Monthly mean (720h) |
Breakthrough (B) | 24 h | ||
Threshold (T) | 720 h | ||
| |||
Accuracy | Goal (G) | 3 % | Merged SLSTR product vs. reference datasets: SLSTR equal area grid: -0.04 % (07/2022 – 12/2023) CLARA-A3: -3.64 % (10/2018 – 12/2023) MODIS: -6.82 % (10/2018 – 12/2023) ERA5: -2.06 % (10/2018 – 12/2023) CALIPSO: -1.47 % (10/2018 – 06/2023) |
Breakthrough (B) | 6 % | ||
Threshold (T) | 12 % | ||
Table 4-2: Results of evaluation against GCOS requirements for SLSTR CTH/CTT
Products | Requirement | Values | Outgoing Longwave Radiation |
Horizontal Resolution | Goal (G) | 25 km |
Roughly 55 km at the equator
|
Breakthrough (B) | 100 km | ||
Threshold (T) | 500 km | ||
| |||
Temporal Resolution | Goal (G) | 1 h |
Monthly mean (720h) |
Breakthrough (B) | 24 h | ||
Threshold (T) | 720 h | ||
| |||
Accuracy | Goal (G) | 0.3 km / 2 K | Merged SLSTR product vs. reference datasets:
SLSTR (EA grid): 0.00 km / -0.01 K (07/2022 – 12/2023) CLARA-A3: -2.52 km / 18.94 K (10/2018 – 12/2023) CALIPSO: -3.74 km / (10/2018 – 06/2023) |
Breakthrough (B) | 0.6 km / 4 K | ||
Threshold (T) | 1.2 km / 8 K | ||
Table 4-3: Results of evaluation against GCOS requirements for SLSTR IWP/LWP
Products | Requirement | Values | Outgoing Longwave Radiation |
Horizontal Resolution | Goal (G) | 25 km |
Roughly 55 km at the equator
|
Breakthrough (B) | 100 km | ||
Threshold (T) | 500 km | ||
| |||
Temporal Resolution | Goal (G) | 1 h |
Monthly mean (720h) |
Breakthrough (B) | 24 h | ||
Threshold (T) | 720 h | ||
| |||
Accuracy | Goal (G) | 0.05 kg/m² | Merged SLSTR product vs. reference datasets:
SLSTR (EA grid): 0.00 kg/m² / 0.00 kg/m² (07/22 – 12/23) CLARA-A3: 0.11 kg/m² / 0.05 kg/m² (10/18 – 12/23) ERA5: 0.17 kg/m² / 0.06 kg/m² (10/18 – 12/23) MODIS: 0.14 kg/m² / 0.09 kg/m² (10/18 – 12/23) |
Breakthrough (B) | 0.1 kg/m² | ||
Threshold (T) | 0.2 kg/m² | ||
This document has been produced with funding by the European Union in the context of the Copernicus Climate Change Service (C3S), operated by the European Centre for Medium-Range Weather Forecasts on behalf on the European Union (Contribution Agreement signed on 22/07/2021).
All information in this document is provided "as is" and no guarantee of warranty is given that the information is fit for any particular purpose. The users thereof use the information at their sole risk and liability. For the avoidance of all doubt, the European Commission and the European Centre for Medium-Range Weather Forecasts have no liability in respect of this document, which is merely representing the author's view.
