Contributors: E. Carboni (UKRI-STFC RAL Space), G.E. Thomas (UKRI-STFC RAL Space)
History of modifications
List of datasets covered by this document
Related documents
Acronyms
Scope of the document
This document provides a description of the product validation methodology for the global Cloud Properties Climate Data Record (CDR). These products are brokered to (in case of (A)ATSR) or produced for the Climate Data Store (in the case of SLSTR) by the Copernicus Climate Change Services (C3S).
ESA’s Cloud_cci ATSR2-AATSR version 3.0 (Level-3C) dataset, produced by STFC RAL Space from the second Along-Track Scanning Radiometer (ATSR-2) on board the second European Remote Sensing Satellite (ERS-2) spanned 1995-2003, the Advanced ATSR (AATSR) on board ENVISAT spanned 2002-2012. In addition the SLSTR on board of Sentinel-3 spans from 2017 to present and forms the ICDR. The validation for the ICDR is over the period from January 2017 to December 2019. The retrieval algorithm is presented in [D2] and the validation methodology refers to the Cloud_cci Product Validation and Intercomparison Report [D1]. The same methodology is applied to SLSTR dataset.
Executive Summary
The ESA Climate Change Initiative (CCI) Cloud Properties Climate Data Record (CDR) is a brokered product from the ESA Cloud_cci project, while the extension Interim CDR (ICDR) produced from the Sea and Land Surface Temperature Radiometers (SLSTR) is produced specifically for C3S. The product is generated by STFC RAL Space, using the Community Cloud for Climate (CC4CL) processor, based on the Optimal Retrieval of Aerosol and Cloud (ORAC) algorithm.
The Cloud_cci dataset comprises 17 years (1995-2012) of satellite-based measurements derived from the Along Track Scanning Radiometers (ATSR-2 and AATSR) on board the ESA second European Research Satellite (ERS-2) and ENVISAT. This TCDR is partnered with the ICDR produced from the Sentinel-3A SLSTR, beginning in 2017, and Sentinel-3B SLSTR beginning in October 2018.
The TCDR and ICDR provide level-3 data, monthly means on a regular global latitude-longitude grid (with a resolution of 0.5 0.5) and daily data (with a resolution of 0.10.1) and includes these products: Cloud Fractional Cover (CFC), Cloud Phase (water/ice), Cloud Optical Thickness (COT), Cloud particle Effective Radius (CER), Liquid/Ice Water Path (LWP/IWP), and Cloud Top Pressure (CTP), Height (CTH) and Temperature (CTT).
1. Validated products
The Cloud_cci ATSR2-AATSR Cloud Properties CDR, version 3.0, is brokered to the CDS from the ESA Cloud_cci project and produced by STFC RAL. The SLSTR ICDR, version 3.x, is supplied to the CDS via the same route and uses the same processing software and infrastructure as the TCDR.
These Cloud Properties datasets from polar orbiting satellites consist of: Cloud Fractional Cover (CFC), Cloud Top Pressure (CTP), Cloud Top Height (CTH), Cloud Top Temperature (CTT), Cloud Effective Radius (CER), Cloud Optical Thickness (COT), Liquid Water Path (LWP), Ice Water Path (IWP) and Cloud Water Path (CWP).
The datasets cover the period from June 1995 to April 2012 (TCDR) of satellite-based measurements derived from ATSR2 and AATSR onboard the polar orbiting ERS-2 and ENVISAT respectively, and the period from January 2017 onwards using the SLSTR measurements (ICDR).These are level 3 products (daily and monthly means) on a regular global latitude-longitude grid (with 0.1° x 0.1° resolution for the daily mean files and 0.5° x 0.5° resolution for the monthly mean files).
ESA’s Cloud_cci dataset on cloud properties, version 3, is the Climate Data Record used for generation of the brokered cloud properties dataset. (DOI: https://doi.org/10.5676/DWD/ESA_Cloud_cci/ATSR2-AATSR/V003).
The Cloud_cci dataset can be downloaded here: https://climate.esa.int/en/projects/cloud/data/. The SLSTR based ICDR extends the coverage, with a five year gap, from 2017 onwards and is only available through Copernicus Climate Data Store (CDS).
Poulsen et al. (2019) [D3] is the paper describing the TCDR dataset that includes cloud products as well as Surface Radiation Budget and Earth Radiation Budget products.
The Product Validation and Intercomparison Report [D1] is the document with validation and intercomparison of cloud properties against other satellite and ground datasets.
2. Description of validating datasets
The Cloud Properties TCDR from ATSR2 and AATSR instruments are validated against CALIOP for cloud fractional cover, and cloud top height.
The cloud top pressure, cloud optical thickness and cloud effective radius (for both liquid and ice cloud) are compared against MODIS Collection 6.1 Terra.
Liquid water path is validated against AMSR-E products and ice water path is validated against the DARDAR IWP product.
We used the CALIOP level-2 1 km and 5 km cloud layer data record versions 3-01, 3-02 and 3-30 ( (https://www-calipso.larc.nasa.gov/) for the validation purpose of cloud fraction and cloud top height.
The MODIS (or Moderate Resolution Imaging Spectroradiometer) 6.1 Terra monthly datasets are available here: https://modis-atmosphere.gsfc.nasa.gov/products/monthly. For the validation we used the data from MODIS Collection 6.1 Terra (dx.doi.org/10.5067/MODIS/MOD08_M3.061).
The Advanced Microwave Scanning Radiometer – EOS (AMSR-E) LWP products can be found here: https://nsidc.org/data/ae_ocean/
The DARDAR data has been downloaded from the University of Lille site:
https://www.icare.univ-lille.fr/dardar/
More details on the datasets used for the validation are described in [D1] section 2.4.2 and Annex A.
3. Description of product validation methodology
The validation strategy is described in [D1] section 2.4. We use the bias, i.e. mean difference between Cloud_cci and the reference data, as the metric for accuracy. The bias corrected root mean squared error (bc-RMSE) is used to express the precision of CDR compared to a reference data record. The stability is the variation of the bias over a multi-annual time period.
Bias (accuracy): | Mean difference between Cloud_cci and reference data |
bc-RMSE (precision): | Bias corrected root mean squared error to express the precision of Cloud_cci compared to a reference data record |
Stability: | The variation of the bias over a multi-annual time period. |
TCDR evaluation is divided into: (i) validation against high quality and satellite-based reference observations (CALIOP, DARDAR and AMSR-E) and (ii) an intercomparison with well-established, satellite-based cloud datasets of similar kind (MODIS).
ICDR data are only validated against MODIS dataset following the same methodology described in section 4.1 of [D1].
3.1 Cloud Fractional Cover (CFC)
Cloud Fractional Cover (CFC) from TCDR is validated against SYNOP in [D1] section 3.2, validated against CALIOP in [D1] section 3.1.1 and compared with MODIS in [D1] section 4.1.1. The validation methodology can be found in the corresponding sections.
ICDR CFC is compared against MODIS using the same methodology described in [D1] section 4.4.1.
3.2 Cloud Top Height (CTH)
Cloud Top Height is validated against the CALIOP data and the validation methodology is described in [D1] section 3.1.3.
ICDR CTH is indirectly validated trough CTP comparison with MODIS.
3.3 Cloud Top Pressure (CTP), Cloud Optical Thickness (COT) and Cloud Effective Radius (CER)
For the TCDR these parameters are compared against MODIS Collection 6.1. Cloud Top Pressure (CTP) is compared in [D1] section 4.1.2, Cloud Effective Radius (CER) is compared in [D1] section 4.1.5 and 4.1.6 and the Cloud Optical Thickness (COT), is compared in [D1] section 4.1.3 and 4.1.4 for the liquid and ice cloud phase. The comparison methodology is described in the corresponding section introduction, 4.1 [D1]. The ICDR uses the same methodology as the TCDR.
3.4 Cloud Liquid Water Path (LWP) and cloud Ice Water Path (IWP)
For the TCDR, Cloud Liquid Water Path (LWP) is validated in [D1] section 3.1.4 against AMSR-E products and compared with MODIS Collection 6.1 in [D1] section 4.1.7. The validation and comparison methodology is described in the mentioned sections.
For the TCDR, Cloud Ice Water Path (IWP) is validated against DARDAR IWP product in [D1] section 3.1.5 and compared with MODIS Collection 6.1 in section 4.1.8. The validation and comparison methodology can be found in the corresponding sections.
For the ICDR, data re compared with MODIS only, according to the methodology described in [D1] section 4.1.
4. Summary of validation results
The validation results for TCDR are provided in [D1] section 7, together with recommendation for use. Here is an extract from table 7-2 in [D1]:
Comments from [D1] | |||
Cloud fractional cover | Accuracy |
| Level-2 validation against CALIOP |
|
| Values taken from Table 4-2 and Table 4-13 (L3C comparisons to MODIS C6.1) | |
Cloud top height/ pressure | Accuracy | 0.12km (liquid cloud) | Level-2 validation against CALIOP |
Stability (per decade) |
| Values taken from Table 4-4 and Table 4-15 (L3C comparisons to MODIS C6.1) | |
Liquid cloud optical depth | Accuracy | n/v | No validation possible due to a lack of reliable reference data. through LWP and IWP validation |
Stability (per decade) |
| Values taken from Table 4-5 and Table 4-16 divided by mean MODIS C6.1 Terra COTliq (13) (L3C comparisons to MODIS C6.1) | |
|
|
| No validation possible due to a lack of reliable reference data. through LWP and IWP validation |
|
| Values taken from Table 4-6 and Table 4-17 divided by mean MODIS C6.1 Terra COTice (10) (L3C comparisons to MODIS C6.1) | |
|
|
| Level-2 validation against AMSR-E (Figure 3-1) |
|
| Values taken from Table 4-9 and Table 4-20 divided by mean MODIS C6 LWP (123g/m²) (L3C comparisons to MODIS C6) | |
Ice water path | Accuracy | -39.9% | Level-2 validation against DARDAR |
|
| Values taken from Table 4-10 and Table 4-21 divided by mean MODIS C6 IWP (208g/m²) (L3C comparisons to MODIS C6) | |
|
|
| No validation possible due to a lack of reliable reference data. through LWP and IWP validation |
|
| Values taken from Table 4-7 and Table 4-18 (L3C comparisons to MODIS C6) | |
|
|
| No validation possible due to a lack of reliable reference data. through LWP and IWP validation |
|
| Values taken from Table 4-8 and Table 4-19 (L3C comparisons to MODIS C6) |
Intercomparison (using the monthly mean data from January 2017 to December 2019) of ICDR products with MODIS present biases consistent with values found through the TCDR MODIS comparison (D1, section 4.1) for cloud fractional cover (- 5 %), cloud top pressure (-15 hPa) and liquid water path (-15.0 g/m2), but higher values for ice water path, with global average bias of -31 g/m3 for SLSTR-A and -46 g/m3 for SLSTR-B (TCDR IWP bias with MODIS was -29 g/m3).
This document has been produced in the context of the Copernicus Climate Change Service (C3S).
The activities leading to these results have been contracted by the European Centre for Medium-Range Weather Forecasts, operator of C3S on behalf of the European Union (Delegation agreement signed on 11/11/2014). All information in this document is provided "as is" and no guarantee or warranty is given that the information is fit for any particular purpose.
The users thereof use the information at their sole risk and liability. For the avoidance of all doubt , the European Commission and the European Centre for Medium - Range Weather Forecasts have no liability in respect of this document, which is merely representing the author's view.
