Contributors: E. Carboni (UKRI-STFC RAL Space), G.E. Thomas (UKRI-STFC RAL Space)

Issued by: STFC RAL Space (UKRI-STFC) / Elisa Carboni

Date: 09/02/2023

Ref: C3S2_D312a_Lot1.1.1.2-v4.0_202302_PQAD_CCICloudProperties_v1.1

Official reference number service contract: 2021/C3S2_312a_Lot1_DWD/SC1

Table of Contents

History of modifications

Version

Date

Description of modification

Chapters / Sections

V1.0

22/07/2022

Brought forward from previous phase of C3S. Updated to include mention of merged Sentinel-3A and -3B level-3 products

All

V1.1

09/02/2023

Revised and issued version

all

List of datasets covered by this document

Deliverable ID

Product title

Product type (CDR, ICDR)

Version number

Delivery date

D3.3.17-v3.0

ECV Cloud properties brokered from ESA's Cloud_cci ATSR-AATSRv3 dataset

CDR

V3.0

30/04/2020

D3.3.18-v3.x

ECV Cloud properties derived from SLSTR

ICDR

V3.1

30/11/2020 - 30/09/2021

D2.1.1-P1/2
D2.1.3-P1

ECV Cloud Properties derived from SLSTR

ICDR

V3.1.1
V4.0

31/05/2022 - onward

Related documents

Reference ID

Document

D1

Product Validation and Intercomparison Report (PVIR), v6.0. ESA Cloud_cci.

https://climate.esa.int/media/documents/Cloud_Product-Validation-and-Intercomparison-Report-PVIR_v6.0.pdf

Last accessed on 16/05/2023

D2

Algorithm Theoretical Basis Document, v.6.2. ESA Cloud_cci.

https://climate.esa.int/media/documents/Cloud_Algorithm-Theoretical-Baseline-Document-ATBD_v6.2.pdf 

Last accessed on 16/05/2023

D3

Poulsen, C. A., McGarragh, G. R., Thomas, G. E., Stengel, M., Christensen, M. W., Povey, A. C., Proud, S. R., Carboni, E., Hollmann, R., and Grainger, R. G.: Cloud{}{_}cci ATSR-2 and AATSR data set version 3: a 17-year climatology of global cloud and radiation properties, Earth Syst. Sci. Data, 12, 2121–2135, 2020.  

https://doi.org/10.5194/essd-12-2121-2020

Last accessed on 16/05/2023

D4

Carboni, E. (2022) C3S Cloud Properties

Service: Product Quality Assessment Report. Copernicus Climate Change Service,

Document ref. C3S2_D312a_Lot1.2.1.5-v4.0_202304_PQAR_CCICloudProperties_v1.1

CP CCI ICDR: Product Quality Assessment Report

Last accessed on 16/05/2023

Acronyms

Acronym

Definition

AATSR

Advanced Along-Track Scanning Radiometer

AMSR-E

Advanced Microwave Scanning Radiometer - EOS

ATBD

Algorithm Theoretical Basis Document

ATSR

Along-Track Scanning Radiometer

C3S

Copernicus Climate Change Service

CALIOP

Cloud-Aerosol Lidar with Orthogonal Polarization

CCI

Climate Change Initiative

CDR

Climate Data Record

CDS

Climate Data Store

CER

Cloud Effective Radius

CFC

Cloud Fractional Cover

COT

Cloud Optical Thickness

CTH

Cloud Top Height

CTP

Cloud Top Pressure

CTT

Cloud Top Temperature

CWP

Cloud Water Path

DARDAR

raDAR/liDAR

ENVISAT

Environmental Satellite

ERS

European Research Satellite

ICDR

Interim CDR

IWP

Ice Water Path

LWP

Liquid Water Path

MODIS

Moderate Resolution Imaging Spectroradiometer

RAL

Rutherford Appleton Laboratory

RMSE

Root Mean Squared Error

SLSTR

Sea and Land Surface Temperature Radiometer

STFC

Science and Technology Facilities Council

SYNOP

Surface synoptic observations

TCDR

Thematic Climate Data Record

List of tables

Table 1-1: Bias of TCDR and ICDR cloud properties estimate in comparison with MODIS from [D4].

Table 4-1: Achieved Cloud_cci accuracy

List of figures

Figure 4-1: CFC, CTP, LWP and IWP from SLSTR (ICDR dataset) for March 2017

Figure 4-2: CFC, CTP, LWP and IWP from MODIS dataset for March 2017

General definitions

The “CCI product family” Climate Data Record (CDR) consists of two parts. The ATSR2-AATSR Cloud Properties CDR is formed by a TCDR brokered from the ESA Cloud_cci project and an ICDR derived from the SLSTR on board of Sentinel-3. ICDR uses the same processing and infrastructure as the TCDR. Both TCDR and ICDR data have been produced by STFC RAL space.

Bias (accuracy): Mean difference between TCDR/ICDR and reference data

\( b=\frac{\sum_{i=1}^N (p_i - r_i)}{N} \ \ (Eq. 1) \)

Where: pi is the CDR product, b is the mean bias and ri is the equivalent value from the reference dataset. N is the number of observations.

bc-RMSE (precision): Bias corrected root mean squared error to express the precision of TCDR/ICDR compared to a reference data record

\( bc- RMSE=\sqrt{\frac{\sum_{i=1}^N ((p-b)-r)^2}{N}} \ \ (Eq. 2) \)

Where: pi is the CDR product, b is the mean bias and ri is the equivalent value from the reference dataset. N is the number of observations.

Stability: The variation of the bias over a multi-annual time period

Table 1: Summary of variables and definitions

Variables

Abbreviation

Definition

Cloud mask / Cloud fraction

CMA/

CFC

A binary cloud mask per pixel (L2) and from there derived monthly total cloud fractional coverage (L3C)

Cloud optical thickness

COT

The line integral of the absorption extinction coefficient (at 0.55μm wavelength) along the vertical in cloudy pixels.

Cloud effective radius

CER

The area-weighted radius of the cloud droplet and crystal particles, respectively.

Cloud top pressure/

height/

temperature

CTP/

CTH/

CTT

 

The air pressure [hPa] /height [m] /temperature [K] of the uppermost cloud layer that could be identified by the retrieval system.

Cloud liquid water path/

Ice water path

 

LWP/

IWP

 

The vertical integrated liquid/ice water content of existing cloud layers; derived from CER and COT. LWP and IWP together represent the cloud water path (CWP)

Table 2: Definition of processing levels

Processing level

Definition

Level-1b

The full-resolution geolocated radiometric measurements (for each view and each channel), rebinned onto a regular spatial grid.

Level-2 (L2)

Retrieved cloud variables at full input data resolution, thus with the same resolution and location as the sensor measurements (Level-1b).

Level-3C (L3C)

Cloud properties of Level-2 orbits of one single sensor combined (averaged) on a global spatial grid. Both daily and monthly products provided through C3S are Level-3C.

Table 3: Definition of various technical terms used in the document

Term

Definition

Brokered product

The C3S Climate Data Store (CDS) provides both data produced specifically for C3S and so-called brokered products. The latter are existing products produced under an independent programme or project which are made available through the CDS.

Climate Data Store (CDS)

The front-end and delivery mechanism for data made available through C3S.

Near-real-time (NRT)

Data which is provided within a short time window (often taken to be three hours, but there is no fixed definition) of the measurement. NRT data is often supplanted by a subsequent data stream, which is subjected to more rigorous checking data quality.

Radiative transfer

The mathematical modelling of the interaction of electromagnetic radiation with some medium – in this case solar and thermal-infrared radiation passing through the Earth’s atmosphere.

Retrieval

A numerical data analysis scheme which uses some form of mathematical inversion to derive physical properties from some form of measurement. In this case, the derivation of cloud properties from satellite measured radiances.

Forward model

A deterministic model which predicts the measurements made of a system, given its physical properties. The forward model is the function which is mathematically inverted by a retrieval scheme. In this case, the forward model predicts the radiances measured by a satellite instrument as a function of atmospheric and surface state, and cloud properties.

TCDR

It is a consistently-processed time series of a geophysical variable of sufficient length and quality.

ICDR

An Interim Climate Data Record (ICDR) denotes an extension of TCDR, processed with a processing system as consistent as possible to the generation of TCDR.

CDR

A Climate Data Record (CDR) is defined as a time series of measurements with sufficient length, consistency, and continuity to determine climate variability and change.

Scope of the document

This document provides a description of the product validation methodology for the global Cloud Properties Climate Data Record (CDR). This CDR comprises inputs from two sources: (i) brokered products from the Cloud Climate Change Initiative (ESA’s Cloud_cci), namely those coming from processing of the Advanced Along-Track Scanning Radiometer (A)ATSR) data  and (ii) those produced under this contract for the Climate Data Store, specifically those coming from processing of the Sea and Land Surface Temperature Radiometers (SLSTR). The Thematic Climate Data Record (TCDR) is the product brokered from the European Space Agency Cloud Climate Change Initiative (ESA’s Cloud_cci) ATSR2-AATSR version 3.0 (Level-3C) dataset.

In addition, the Interim Climate Data Record (ICDR) is the product derived from the SLSTR on board of Sentinel-3 and spans the period  from 2017 to present. Validation of this SLSTR derived product for the period from January 2017 to December 2021 is described in this document. This document summarizes and refers to the methodology presented in the Cloud_cci Product Validation and Intercomparison Report [D1], used in the validation of the TCDR product. The same methodology is applied to the ICDR dataset.

Executive Summary

The ESA Climate Change Initiative (CCI) Cloud Properties Climate Data Record (CDR) is a brokered product from the ESA Cloud_cci project, while the extension Interim CDR (ICDR) produced from the Sea and Land Surface Temperature Radiometers (SLSTR) is produced specifically for C3S. The product is generated by STFC RAL Space, using the Community Cloud for Climate (CC4CL) processor, based on the Optimal Retrieval of Aerosol and Cloud (ORAC) algorithm.

The Cloud_cci dataset comprises 17 years (1995-2012) of satellite-based measurements derived from the Along Track Scanning Radiometers (ATSR-2 and AATSR) on board the ESA second European Research Satellite (ERS-2) and ENVISAT. This TCDR is partnered with the ICDR produced from the Sentinel-3A SLSTR, beginning in 2017, and Sentinel-3B SLSTR beginning in October 2018. In addition to individual products from each Sentinel-3 platform, a combined product that averages data from both SLSTR instruments into single daily and monthly means will also be provided.

The TCDR and ICDR provide level-3 data, monthly means on a regular global latitude-longitude grid (with a resolution of 0.5°´ 0.5°) and daily data (with a resolution of 0.1°´0.1°) and includes these products: Cloud Fractional Cover (CFC), Cloud Phase (water/ice), Cloud Optical Thickness (COT), Cloud particle Effective Radius (CER), Liquid/Ice Water Path (LWP/IWP), and Cloud Top Pressure (CTP), Height (CTH) and Temperature (CTT).

This document is divided into different sections:

  • the first section presents a brief description of the Cloud Properties CDR products together with references for further information;
  • the second section presents the datasets used to estimate the accuracy of the CDR Cloud Properties dataset;
  • the third section presents the methodology used for the validation and is divided in different subsections that describe the different parameters: CFC, CTH, CTP, COT, CER, LWP and IWP.

1. Validated products

The Cloud Properties CDR is formed by a TCDR brokered from the ESA Cloud_cci project and an ICDR derived from the SLSTR on board of Sentinel-3. Both TCDR and ICDR data have been produced by STFC RAL space.

The SLSTR ICDR, both from the individual instruments (version 3.0) and combining both in a single product (version 4.0), is supplied to the CDS via the same route and uses the same processing software and infrastructure as the TCDR. The retrieval algorithm is described in [D2]

These Cloud Properties datasets from polar orbiting satellites consist of: Cloud Fractional Cover (CFC), Cloud Top Pressure (CTP), Cloud Top Height (CTH), Cloud Top Temperature (CTT), Cloud Effective Radius (CER), Cloud Optical Thickness (COT), Liquid Water Path (LWP), Ice Water Path (IWP) and Cloud Water Path (CWP).

The datasets cover the period from June 1995 to April 2012 (TCDR) of satellite-based measurements derived from ATSR2 and AATSR onboard the polar orbiting ERS-2 and ENVISAT respectively, and the period from January 2017 onwards using the SLSTR measurements (ICDR), with Sentinel-3b and combined data becoming available from October 2018. These are level 3 products (daily and monthly means) on a regular global latitude-longitude grid (with 0.1° x 0.1° resolution for the daily mean files and 0.5° x 0.5° resolution for the monthly mean files). Table 1-1 report the bias values from [D4].

ESA’s Cloud_cci dataset on cloud properties, version 3, is the Climate Data Record used for generation of the brokered cloud properties dataset1.

The Cloud_cci dataset can be downloaded here https://climate.esa.int/en/projects/cloud/data/. The SLSTR based ICDR extends the coverage, with a five year gap, from 2017 onwards and is only available through Copernicus Climate Data Store (CDS).

The TCDR dataset that includes cloud products as well as Surface Radiation Budget and Earth Radiation Budget products are described by Poulsen et al. (2019) [D3]. 

Table 1-1: Bias of TCDR and ICDR cloud properties estimate in comparison with MODIS from [D4].

Parameters

TCDR (2003-2011) bias

ICDR (2017-2021) bias

SLSTR-A

ICDR (2019-2021) bias

SLSTR-B

CFC

- 8.1%

- 6

-6%

CTP

-25 hPa

-17 hPa

-17 hPa

LWP

-17.3 g/m2

-14.0 g/m2

-6 g/m2

IWP

-28.8 g/m2

-35 g/m2

-54 g/m2

2. Description of validating datasets

The Cloud Properties TCDR from ATSR2 and AATSR instruments are validated against CALIOP for cloud fractional cover, and cloud top height. We used the CALIOP level-2 1 km and 5 km cloud layer data record versions 3-01, 3-02 and 3-301 for validation of cloud fraction and cloud top height.

The Cloud-Aerosol Lidar and Infrared Pathfinder Satellite Observation (CALIPSO) satellite was launched in April 2006 together with CloudSat. The satellite carries the Cloud-Aerosol Lidar with Orthogonal Polarization (CALIOP) and the first data became available in August 2006. CALIOP provides detailed profile information about cloud and aerosol particles and corresponding physical parameters.

The cloud top pressure, cloud optical thickness and cloud effective radius (for both liquid and ice cloud) are compared against MODIS Collection 6.1 Terra. The MODIS (or Moderate Resolution Imaging Spectroradiometer) 6.1 Terra monthly datasets are available here2. For the validation we used the data from MODIS Collection 6.1 Terra (dx.doi.org/10.5067/MODIS/MOD08_M3.061).

Terra passes from north to south across the equator in the morning (local solar time 10:30). MODIS Terra and MODIS Aqua are viewing the entire Earth's surface every 1 to 2 days, acquiring data in 36 spectral bands or groups of wavelengths. The MODIS observation period started in 2000. We have used the level-3 MODIS gridded atmosphere monthly global products - MOD08_M3 (Terra). They contain monthly 1° × 1° grid average values of atmospheric parameters related to atmospheric aerosol particle properties, total ozone burden, atmospheric water vapour, cloud optical and physical properties, and atmospheric stability indices. Statistics are sorted into 1° × 1° cells on an equal-angle grid that spans a (calendar) monthly interval and are then summarized over the globe.

Liquid water path is validated against AMSR-E products and ice water path is validated against the DARDAR IWP product. The Advanced Microwave Scanning Radiometer – EOS (AMSR-E) LWP products can be found here3 The DARDAR data has been downloaded from the University of Lille site4.

Passive microwave imagers, such as the Advanced Microwave Scanning Radiometer – EOS (AMSR-E), can be used to retrieve column-integrated liquid water along with water vapour and surface wind speed. AMSR-E is a dual-polarization conical-scanning passive microwave radiometer with 12 channels ranging from 6.9 to 89 GHz. This instrument was designed to measure cloud properties, sea surface temperature and surface water, ice and snow. Because the microwave (MW) channels usually fully penetrate clouds, they provide a direct measurement of the total liquid (but not solid) cloud condensate amount.

DARDAR is a combined product based on measurement by CALIOP lidar and CPR onboard CloudSat. CPR is a nadir-looking cloud profiling radar sensing the atmosphere from above at 94 GHz. The DARDAR product has the vertical resolution of CALIOP (30/60 m) and a horizontal resolution given by the radar footprint (700m).

More details on the datasets used for the validation are described in [D1] section 2.4.2 and Annex A of [D1].

3. Description of product validation methodology

The validation strategy is described in [D1] section 2.4. We use the bias, i.e. mean difference between Cloud_cci and the reference data, as the metric for accuracy. The bias corrected root mean squared error (bc-RMSE) is used to express the precision of CDR compared to a reference data record. The stability is the variation of the bias over a multi-annual time period.

Bias (accuracy):

Mean difference between Cloud_cci and reference data

bc-RMSE (precision):

Bias corrected root mean squared error to express the precision of Cloud_cci compared to a reference data record

Stability:

The variation of the bias over a multi-annual time period.

TCDR evaluation is divided into:

(i) validation against high quality and satellite-based reference observations (CALIOP, DARDAR and AMSR-E)

(ii) an intercomparison with well-established, satellite-based cloud datasets of similar kind (MODIS).

ICDR data are only validated against MODIS dataset following the same methodology described in section 4.1 of [D1].

3.1 Cloud Fractional Cover (CFC)

Cloud Fractional Cover (CFC) from TCDR is validated against SYNOP in [D1] section 3.2, validated against CALIOP in [D1] section 3.1.1 and compared with MODIS in [D1] section 4.1.1. The validation methodology can be found in the corresponding sections.

The ICDR CFC is compared against MODIS using the same methodology described in [D1] section 4.4.1.

3.2 Cloud Top Height (CTH)

Cloud Top Height is validated against the CALIOP data and the validation methodology is described in [D1] section 3.1.3.

The ICDR CTH is indirectly validated trough CTP comparison with MODIS.

3.3 Cloud Top Pressure (CTP), Cloud Optical Thickness (COT) and Cloud Effective Radius (CER)

For the TCDR these parameters are compared against MODIS Collection 6.1.

  • The Cloud Top Pressure (CTP) is compared in [D1] section 4.1.2
  • The Cloud Optical Thickness (COT), is compared in [D1] section 4.1.3 and 4.1.4 for the liquid and ice cloud phase.
  • The Cloud Effective Radius (CER) is compared in [D1] section 4.1.5 and 4.1.6.

The comparison methodology is described in the corresponding section introduction, 4.1 [D1]. The ICDR uses the same methodology as the TCDR.

3.4 Cloud Liquid Water Path (LWP) and cloud Ice Water Path (IWP)

For the TCDR, the validation of the Cloud Liquid Water Path (LWP) against AMSR-E products is presented in [D1] section 3.1.4. The  comparison with MODIS Collection 6.1 is presented in [D1] section 4.1.7. The validation and comparison methodology are described in the mentioned sections.

For the TCDR, the validation of the Cloud Ice Water Path (IWP) against the DARDAR IWP product is presented in [D1] section 3.1.5. The comparison with MODIS Collection 6.1 is presented in section 4.1.8. The validation and comparison methodology can be found in the corresponding sections.

For the ICDR, data are compared with MODIS Collection 6.1 only, according to the methodology described in [D1] section 4.1.

4. Summary of validation results

The validation results for TCDR are provided in [D1] section 7, together with recommendation for use. Table 4-1 is an extract from table 7-2 in [D1]:

Table 4-1: Achieved Cloud_cci accuracy

Cloud Parameter



 Comments from [D1]

Cloud fractional cover

Accuracy


-5.1 %

Level-2 validation against CALIOP



Stability (per decade)


-0.52%

Values taken from Table 4-2 and Table 4-13 (L3C comparisons to MODIS C6.1)

Cloud top height/ pressure

Accuracy

0.12km (liquid cloud)
-1.76km (ice cloud)

Level-2 validation against CALIOP


Stability (per decade)


0.45hPa

Values taken from Table 4-4 and Table 4-15 (L3C comparisons to MODIS C6.1)

Liquid cloud optical depth

Accuracy

n/v

No validation possible due to a lack of reliable reference data. through LWP and IWP validation


Stability (per decade)


-0.03%

Values taken from Table 4-5 and Table 4-16 divided by mean MODIS C6.1 Terra COTliq (13) (L3C comparisons to MODIS C6.1)


Ice cloud optical depth


Accuracy


n/v

No validation possible due to a lack of reliable reference data. through LWP and IWP validation



Stability (per decade)


-0.05%

Values taken from Table 4-6 and Table 4-17 divided by mean MODIS C6.1 Terra COTice (10) (L3C comparisons to MODIS C6.1)


Liquid water path


Accuracy


-2.4%

Level-2 validation against AMSR-E (Figure 3-1)



Stability (per decade)


-0.06%

Values taken from Table 4-9 and Table 4-20 divided by mean MODIS C6 LWP (123g/m²) (L3C comparisons to MODIS C6)

Ice water path

Accuracy

-39.9%

Level-2 validation against DARDAR
(Figure 3-2)



Stability (per decade)


-0.04%

Values taken from Table 4-10 and Table 4-21 divided by mean MODIS C6 IWP (208g/m²) (L3C comparisons to MODIS C6)


Cloud liquid effective particle radius


Accuracy


n/v

No validation possible due to a lack of reliable reference data. through LWP and IWP validation



Stability (per decade)


-0.96μm

Values taken from Table 4-7 and Table 4-18 (L3C comparisons to MODIS C6)


Cloud ice effective particle radius


Accuracy


n/v

No validation possible due to a lack of reliable reference data. through LWP and IWP validation



Stability (per decade)


-0.33μm

Values taken from Table 4-8 and Table 4-19 (L3C comparisons to MODIS C6) 

Intercomparison (using the monthly mean data from January 2017 to December 2019) of ICDR products with MODIS present biases consistent with values found through the TCDR MODIS comparison (D1, section 4.1) for cloud fractional cover (- 5 %), cloud top pressure (-17 hPa) and liquid water path (-14.0 g/m2 for SLSTR-A and -6 g/m2 for SLSTR-B). 

The intercomparison present higher bias for ice water path, with global average bias of -35 g/m2 for SLSTR-A and -54 g/m2 for SLSTR-B (TCDR IWP bias with MODIS was -29 g/m3).

Figure 4-1 and Figure 4-2 show an example of ICDR monthly products for March 2017 and the equivalent monthly product from MODIS. More detailed description and analysis of the results is available in [D4].

Figure 4-1: CFC, CTP, LWP and IWP from SLSTR (ICDR dataset) for March 2017

Figure 4-2: CFC, CTP, LWP and IWP from MODIS dataset for March 2017

References

Poulsen, C. A., McGarragh, G. R., Thomas, G. E., Stengel, M., Christensen, M. W., Povey, A. C., Proud, S. R., Carboni, E., Hollmann, R., and Grainger, R. G.: Cloud_cci ATSR-2 and AATSR data set version 3: a 17-year climatology of global cloud and radiation properties, Earth Syst. Sci. Data, 12, 2121–2135, 2020, https://doi.org/10.5194/essd-12-2121-2020.

This document has been produced in the context of the Copernicus Climate Change Service (C3S).

The activities leading to these results have been contracted by the European Centre for Medium-Range Weather Forecasts, operator of C3S on behalf on the European Union (Contribution Agreement signed on 22/07/2021). All information in this document is provided “as is” and no guarantee of warranty is given that the information is fit for any particular purpose.

The users thereof use the information at their sole risk and liability. For the avoidance of all doubt, the European Commission and the European Centre for Medium-Range Weather Forecasts have no liability in respect of this document, which is merely representing the author’s view.

Related articles