Last modified on May 30, 2025 12:14

Contributors: G.E. Thomas (UKRI-STFC RAL Space) and R. Shevchuk (Brockmann Consult)

Issued by: STFC RAL Space (UKRI-STFC) / Gareth Thomas and Brockmann Consult / Roman Shevchuk

Date: 12 Jan 2024 

Ref: C3S2_D312a_Lot1.2.1.6_202407_PUGS_ECV_CLD_SLSTR_v1.2_final

Official reference number service contract: 2021/C3S2_312a_Lot1_DWD/SC1

History of modifications

List of datasets covered by this document

Related documents

Acronyms

List of tables

List of figures

General definitions

Table 1: Summary of variables and definitions

Table 2: Definition of processing levels

Table 3: Definition of various technical terms used in the document

Scope of the document

This Product User Guide and Specification (PUGS) document is associated with the Climate Data Store (CDS) catalogue entry: Cloud properties global gridded monthly and daily data from 1979 to present derived from satellite observations. This Climate Data Record (CDR) comprises inputs from three sources: (i) brokered products from the Cloud Climate Change Initiative (ESA’s Cloud_cci), namely those coming from processing of the Advanced Along-Track Scanning Radiometer (A)ATSR data (also named TCDR (Thematic Climate Data Record)), and two Interim Climate Data Records (ICDR) extensions namely (ii) those produced under this contract for the CDS, specifically those coming from processing of the Sea and Land Surface Temperature Radiometers (SLSTR) by Rutherford Appleton Laboratory (RAL) Space and (iii) those produced under this contract for the CDS, specifically those coming from processing of the SLSTR by Brockmann Consult (BC).

Both of the ICDRs mentioned above are produced using the Optimal Retrieval of Aerosol and Cloud (ORAC) algorithm, using the Community Cloud for Climate (CC4CL) processor under the ESA Cloud_cci project. One ICDR was produced by Rutherford Appleton Laboratory (RAL) Space for the period 01/2017 to 06/2022 and the other ICDR by Brockmann Consult (BC) for the period 07/2022 to 12/2023.

This document provides information on how to use these products. It is not part of the official CCI documentation, but produced only in the scope of the brokering to the CDS.

Executive summary

The ESA Climate Change Initiative (CCI) Cloud Properties Climate Data Record (CDR) is a brokered product from the ESA Cloud_cci project, while the extension Interim CDR (ICDR) produced from the Sea and Land Surface Temperature Radiometers (SLSTR) is produced specifically for C3S. The product is generated using the Community Cloud for Climate (CC4CL) processor, based on the Optimal Retrieval of Aerosol and Cloud (ORAC) algorithm. The ICDR product was generated by RAL Space up to 06/2022 and by BC for the period 07/2022 to 12/2023.

The Cloud_cci dataset comprises 17 years (1995-2012) of satellite-based measurements derived from the Along Track Scanning Radiometers (ATSR-2 and AATSR) onboard the ESA second European Research Satellite (ERS-2) and ENVISAT. This TCDR is partnered with the ICDR produced from the Sentinel-3 Sea and Land Surface Temperature Radiometers (SLSTR), beginning in 2017 (Sentinel-3A) and 2018 (Sentinel-3B), when data from each platform became available. In addition to the individual data from each platform, a merged product is provided from 10/2018 onwards.

The TCDR and ICDRs provide both level-3 (monthly means) and level-2 (daily means) on two grids: (1) regular equal angle global latitude-longitude grid and (2) regular equal area global latitude-longitude grid (the second option solely for the merged product from 07/2022 onwards). An overview of the various data producers, satellites, grids and time coverages for the ICDRs is shown in section 1, Figure 1-1.

Both grids have a resolution of 0.5°× 0.5° for monthly means and 0.1°× 0.1° for the daily means. Both grids include these products: cloud fractional cover (CFC), cloud phase (water/ice), cloud optical thickness (COT), cloud particle effective radius (CER), cloud liquid/ice water path (LWP/IWP), and cloud-top pressure (CTP), height (CTH) and temperature (CTT).

These products are brokered to (in the case of (A)ATSR) or produced for the Climate Data Store (in the case of SLSTR) by the Copernicus Climate Change Services (C3S). The respective data products (daily and monthly means) are first described in terms of their input data and a brief overview of the algorithms is provided; their target requirements in the scope of C3S and achieved performances are given; relevant information for usage is provided. The latter comprises geographical grid specifications, the data format, naming conventions, and the acknowledgement policy.

1. Cloud_cci global monthly cloud products

The Cloud_cci global monthly cloud properties product, version 3.0, is brokered to the CDS by the C3S from STFC RAL Space. The SLSTR ICDR until 06/2022, versions 3.1.1 and 4.0, is supplied to the CDS via the same route and uses the same processing software and infrastructure as the TCDR. The SLSTR ICDR, version 4.0 from 07/2022 to 12/2023 is processed on BC infrastructure using the same processing software for Level-2, the additional grid version (equal area) was processed using the Calvalus aggregator for Level-3 step [D6]. Although the format of the brokered data products differs from the official CCI products, the intellectual property rights remain with the Cloud_cci team. In detail, the official daily and monthly files from Cloud_cci contain more than 180 variables, but the specified C3S format splits them into separate parts per ECV. That way, each file has a reduced data amount as well as an increased usability. The landing page for the official CCI products is https://climate.esa.int/en/projects/cloud/. Detailed information on the cloud products and their use are outlined in the CCI Product User Guide [D3].

The product provides estimates of monthly mean global cloud properties over land and ocean from the Along Track Scanning Radiometer ((A)ATSR) series of satellite sensors, on a 0.5°×0.5° latitude-longitude grid from mid-1995 until early 2012. The SLSTR based ICDR extends the coverage, with a five-year gap, from 2017 onwards and is only available through the CDS. Version 3.1.1 of the ICDR provides data from a single SLSTR (flying on board either Sentinel-3a or -3b), while version 4.0 combines data from both SLSTR instruments into a single combined product (starting with the launch of the Sentinel-3b in late 2018). The processing of version 3.1.1 and 4.0 is identical – the sole difference is whether the two instruments are treated separately or as a single data-stream. All monthly means are provided on an equal angle projection. In addition, the product from 07/2022 onwards is provided on an equal area projection as well (see Figure 1-1 for an overview).

Figure 1-1: Overview of data producers, satellites, time coverages and grids for the ICDRs. Products above/below the horizontal black line are produced by RAL Space/Brockmann Consult (BC) respectively, while data from 07/2022 onwards is provided on two different grids.

1.1 Product description

The Cloud_cci product brokered to the CDS consists of cloud properties derived from the (A)ATSR instruments; specifically:

• ATSR-2, which flew on the second European Research Satellite (ERS-2) from June 1995 until January 2008. ERS-2 lost global coverage in June 2003, due to failure of the onboard data storage; Cloud_cci data do not extent beyond this date (thus providing approximately one-year of overlap with the succeeding AATSR product).
• Advanced-ATSR (AATSR), which flew on ENVISAT from May 2002 until the satellite’s failure in April 2012.

When referred to in terms of a continuous data record, these two instruments are commonly described as (A)ATSR. The preceding ATSR-1 instrument lacked the visible-NIR channels provided by (A)ATSR and thus isn’t included in the Cloud_cci product.

The Cloud_cci also produced cloud products from the Advanced Very High Resolution Radiometer (AVHRR) series of instruments and the Moderate Resolution Imaging Spectroradiometer (MODIS), using a common retrieval technique, but only the (A)ATSR products are brokered to the CDS.

The analysis algorithm applied in producing the Cloud_cci products is the Optimal Retrieval of Aerosol and Clouds (ORAC), while the overall data processing system used in the project is referred to as Community Cloud for Climate (CC4CL). The algorithm, processing chain and all input and auxiliary data are described in the C3S ATBD [D1], Cloud_cci ATBD [D2], Sus et al. (2018) and McGarragh et al. (2018).

The brokered TCDR is extended, with a five-year gap, from 2017 with an ICDR derived from the SLSTR instruments, which fly onboard the Copernicus operational Sentinel-3 satellites. Since late 2018, a second Sentinel-3 platform is available, which doubles the spatial coverage of SLSTR and allows for nearly complete global coverage twice a day. The SLSTR product is produced using the same processing chain as the (A)ATSR TCDR.

1.2 Target requirements

The target requirements for cloud observations are laid-out in the Target Requirements and Gap Analysis Document (TRGAD) [D5], but are summarised here. These requirements originate from the WMO Global Climate Observing System (GCOS) initiative, which defines and lays down targets for the observation of essential climate variables (ECVs). It should be noted that GCOS defines ideal targets for Earth observation systems, which are often not attainable using existing or historical observing systems. Table 1-1 provides an overview of the GCOS requirements for cloud properties.

Table 1-1: GCOS targets for cloud properties ECVs. Note that the Cloud_cci doesn't meet the temporal requirements due to the nature of the satellite observations, but exceeds the spatial resolution and aims to achieve the uncertainty and stability requirements.

1.3 Data usage information

Cloud_cci and SLSTR ICDR products are provided in NetCDF (version 4), which are compliant with the conventions CF 1.8 and the NASA Global Change Master Directory (GCMD) Science Keywords vocabulary. Filenames follow the structure:

C3S-312bL1-L3C-MONTHLY-CLOUD-INST_ORAC_EA_PLATFORM_YYYYMM_fvV.V.nc,

where INST and PLATFORM refer to the instrument and platform from which data originates (either ATSR2 and ERS2, or AATSR and ENVISAT for the TCDR, or SLSTR and Sentinel-3a or -3b for the ICDR), EA is added for the additional equal area grid and left out for the equal angle grid, and YYYYMM provides the year and month covered by the monthly mean product, and V.V denotes the product version.

Data are provided as monthly means of cloud properties, as described in Table 1-2, on a regular latitude-longitude grid, with a spacing of 0.5° in both dimensions (thus grid centres lie at -89.75°, -89.25°, -88.75°, …, 89.75° in latitude and -179.75°, -179.25°, -178.75°, …, 179.75° in longitude).

The list of cloud properties, the variable names used within the NetCDF data files and their units are given in Table 1-2.

Table 1-2: Cloud properties included in Cloud_cci TCDR and SLSTR ICDR monthly products available through the CDS.

For lists and details of known issues and limitations of the data, the user is referred to the referenced ATBDs and papers in Section 1.1, as well as the TRGAD [D5]. In particular, it should be noted that the monthly mean is computed using data from sun-synchronous polar-orbiting satellites, with a relatively narrow swath. Thus, it cannot be considered a representation of the mean properties of clouds across the whole month for two reasons:

• The narrow swath of the (A)ATSR instruments means that it requires approximately three days of measurements to provide full global coverage.
• The sun-synchronous orbit used by ERS-2, ENVISAT and Sentinel-3, means that observations are made near two fixed local solar times¹; 10:00 and 22:00 for the descending and ascending halves of the ENVISAT and Sentinel-3 orbit, and 10:30 and 22:30 for ERS-2. Thus, the diurnal cycle is poorly sampled.

1.3.1 Uncertainties

The ORAC retrieval algorithm is an implementation of Optimal Estimation (Rodgers, 2000) and as such provides uncertainty estimates on all retrieved parameters, propagated from the uncertainty in the input satellite measurements (as well as any forward modelling error included in the uncertainty calculations). The propagation of the L2 uncertainty into L3 products, such as those brokered to the CDS, is not straight forward. Users should refer to the Cloud_cci Comprehensive Error Characterisation Report [D4].

Uncertainty and variability in the monthly mean cloud properties is expressed using three separate values, as described by Bennartz (2017):

1. Standard deviation about the mean. This value is the unweighted standard deviation of the L2 retrieval values about the mean value reported for each L3 pixel.

2. Propagated uncertainty in value. The value represents the L2 uncertainty propagated to the L3 pixel under the assumption that the L2 values used in the L3 average are independent and uncorrelated. It is calculated by summing the L2 uncertainties in quadrature and normalizing by the number of observations:

where σ_i are the uncertainties of the N L2 pixels included in the average, and the <> indicates the mean value.

3. Correlated uncertainty in value. If the L2 pixels are assumed to be correlated with each other, the uncertainty in the resulting average can be calculated using: 

where σ²_std is the unweighted standard deviation about the mean and c is the correlation between the L2 pixels.

2. Cloud_cci global daily cloud products

The Cloud_cci global daily cloud properties product, version 3.0, is brokered to the CDS by the C3S from STFC RAL Space. The SLSTR ICDR, versions 3.1.1 and 4.0 are supplied to the CDS via the same route and uses the same processing software and infrastructure as the TCDR. The SLSTR ICDR, version 4.0 from 07/2022 to 12/2023 is processed on BC infrastructure using the same processing software for Level-2, the additional grid version (equal area) was processed using Calvalus aggregator for Level-3 step. Although the format of the brokered data products differs from the official Cloud_cci products, the intellectual property rights remain with the Cloud_cci team. In detail, the official daily and monthly files from Cloud_cci contain more than 180 variables, but the specified C3S format splits them into separated parts per ECV. That way, each file has a reduced data amount as well as an increased usability. The landing page for the official CCI products is https://climate.esa.int/en/projects/cloud/.

The product provides estimates of daily mean global cloud properties over land and ocean from the Along Track Scanning Radiometer ((A)ATSR) series of satellite sensors, on a 0.1°×0.1° latitude-longitude grid from mid-1995 until early 2012. The SLSTR based ICDR extends the coverage, with a five-year gap, from 2017 onwards and is only available through the CDS. Version 3.x of the ICDR provides data from a single SLSTR (flying on board either Sentinel-3a or -3b), while version 4.0 combines data from both SLSTR instruments into a single combined product (starting with the launch of the Sentinel-3b in late 2018). The processing of version 3.x and 4.0 are identical – the sole difference is whether the two instruments are treated separately or as a single data-stream. All monthly means are provided on an equal angle projection. In addition, the product from 07/2022 onwards is provided on an equal area projection as well (see Figure 1-1 for an overview).

2.1 Product description

The Cloud_cci product brokered to the CDS consists of cloud properties derived from the (A)ATSR instruments; specifically:

• ATSR-2, which flew on the second European Research Satellite (ERS-2) from June 1995 until January 2008. ERS-2 lost global coverage in June 2003, due to failure of the onboard data storage; Cloud_cci data do not extend beyond this date (thus providing approximately one-year of overlap with the succeeding AATSR product.
• Advanced-ATSR (AATSR), which flew on ENVISAT from May 2002 until the satellite’s failure in April 2012.

When referred to in terms of a continuous data record, these two instruments are commonly described as (A)ATSR. The preceding ATSR-1 instrument lacked the visible-NIR channels provided by (A)ATSR and thus isn’t included in the Cloud_cci product.

The Cloud_cci also produced cloud properties from the Advanced Very High Resolution Radiometer (AVHRR) series of instruments and the Moderate Resolution Imaging Spectroradiometer (MODIS), using a common retrieval technique, but only the (A)ATSR products are brokered to the CDS.

The analysis algorithm applied in producing the Cloud_cci products is the Optimal Retrieval of Aerosol and Clouds (ORAC), while the overall data processing system used in the project is referred to as Community Cloud for Climate (CC4CL), and these are detailed in the C3S ATBD [D1], Cloud_cci ATBD [D2], Sus et al. (2018) and McGarragh et al. (2018).

The brokered TCDR is extended, with a five-year gap, from 2017 with an ICDR derived from the SLSTR instruments, which fly onboard the Copernicus operational Sentinel-3 satellites. Since late 2018, a second Sentinel-3 platform is available, which doubles the spatial coverage of SLSTR and allows for nearly complete global coverage twice a day. The SLSTR product is produced using the same processing chain as the (A)ATSR TCDR.

2.2 Target requirements

The target requirements for cloud observations are laid-out in the Target Requirements and Gap Analysis Document (TRGAD) [D5], but are summarised here. These requirements originate from the WMO Global Climate Observing System (GCOS) initiative, which defines and lays down targets for the observation of essential climate variables (ECVs). It should be noted that GCOS defines ideal targets for Earth observation systems, which are often not attainable using existing or historical observing systems. Table 2-1 provides an overview of the GCOS requirements for cloud properties.

Table 2-1: GCOS targets for cloud ECVs. Note that the Cloud_cci doesn’t meet the temporal requirements due to the nature of the satellite observations, but exceeds the spatial resolution and aims to achieve the uncertainty and stability requirements.

2.3 Data usage information

Cloud_cci products are provided in NetCDF (version 4), which are compliant with the conventions CF 1.8 and the NASA Global Change Master Directory (GCMD) Science Keywords vocabulary. Filenames follow the structure:

C3S-312bL1-L3C-DAILY-CLOUD-INST_ORAC_EA_PLATFORM_YYYYMMDD_ fvV.V.nc,

where INST and PLATFORM refer to the instrument and platform from which data originates (either ATSR2 and ERS2, or AATSR and ENVISAT for the TCDR, or SLSTR and Sentinel-3a or -3b for the ICDR), EA is added for the additional equal area grid and left out for the equal angle grid, and YYYYMM provides the year and month covered by the monthly mean product, and V.V denotes the product version.

Data are provided as daily means of cloud properties, as described in Table 2-2, on a regular latitude-longitude grid, with a spacing of 0.1° in both dimensions (thus grid centres lie at -89.95°, -89.85°, -89.75°, …, 89.95° in latitude and -179.95°, -179.85°, -179.75°, …, 179.95° in longitude).

Unlike the monthly product, the daily product separates the variables into day and night observations, stored in different variables within the product. In this context “day” measurements are defined as those where the solar zenith angle is low enough that the solar reflectance can be used in the retrieval scheme. In the case of ORAC this means a solar zenith angle of . There are two main reasons for this approach:

1. In a given location and day, observations made by a sun-synchronous satellite during day and night will be separated by approximately 12 hours. By separating the averages into day and night values, we are essentially representing an instantaneous spatial average, rather than combining two instantaneous measurements separated by many hours.
2. Furthermore, if an average of all available data for a given day was taken, some grid-cells would contain data from both day and night observations, some would contain only day and some only night observations. Thus, the resulting product would contain discontinuities which would be solely an artefact of the averaging.

These two factors combine to mean that separate day and night averages are both more geophysically relevant and easier to interpret than a simple average over the whole day.

The list of cloud properties, the variable names used within the NetCDF data files and their units are given in Table 2-2.

Table 2-2: Cloud properties included in Cloud_cci TCDR and SLSTR ICDR daily products available through the CDS.

For lists and details of known issues and limitations of the data, the user is referred to the referenced ATBDs and papers in Section 1.1.

2.3.1 Uncertainties

The ORAC retrieval algorithm is an implementation of Optimal Estimation (Rodgers, 2000) and as such provides uncertainty estimates on all retrieved parameters, propagated from the uncertainty in the input satellite measurements (as well as any forward modelling error included in the uncertainty calculations). The propagation of the L2 uncertainty into L3 products, such as those brokered to the CDS, is not straight forward. Users should refer to the Cloud_cci Comprehensive Error Characterisation Report [D4].

Uncertainty and variability in the daily mean cloud properties is expressed using three separate values, as described by Bennartz (2017, pers. comm.):

1. Standard deviation about the mean. This value is the unweighted standard deviation of the L2 retrieval values about the mean value reported for each L3 pixel.

2. Propagated uncertainty in value. The value represents the L2 uncertainty propagated to the L3 pixel under the assumption that the L2 values used in the L3 average are independent and uncorrelated. It is calculated by summing the L2 uncertainties in quadrature and normalizing by the number of observations: 

where σ_i are the uncertainties of the N L2 pixels included in the average, and the <> indicates the mean value.

3. Correlated uncertainty in value. If the L2 pixels are assumed to be correlated with each other, the uncertainty in the resulting average can be calculated using: 

where σ²_std  is the unweighted standard deviation about the mean and c is the correlation between the L2 pixels.

3. Data access information

The original repository for the CCI formatted L3 data can be accessed through the Cloud_cci homepage https://climate.esa.int/en/projects/cloud/data/, as can all relevant documentation from the project. It should be noted that Cloud_cci L3 data is not in the same format as data brokered to the CDS, however the same L2 retrieval output was used to produce both products. In detail, the official daily and monthly files from Cloud_cci contain more than 180 variables, but the specified C3S format split them into separate parts per ECV. That way, each file has a reduced data amount as well as an increased usability.

Cloud_cci data should also be available through the ESA_cci data portal http://cci.esa.int/data, which provides online data mining, analysis and visualization tools for CCI data through the CCI Toolbox.

Only the(A)ATSR based TCDR is available through the ESA_cci data portal (original provider) as well as the CDS (brokered from CCI within the C3S Project), with SLSTR data being only supplied by the CDS.

3.1 Data access through the CDS

Within C3S, the distribution is through the CDS (https://cds.climate.copernicus.eu/) where documentation created for the inclusion of the data in the CDS, such as this PUGS, is also provided.

3.2 Product ordering process

You need to be registered and logged in to order products in the CDS. A login is provided upon registration, all products in the CDS are delivered free of charge.

References

Bennartz, R., 2017: Error propagation in the generation of gridded satellite datasets, DWD Visiting Scientist Report.

McGarragh, G., C.A. Poulsen, G.E. Thomas, A.C. Povey, O. Sus, S. Stapelberg, C. Schlundt, S. Proud, M.W. Christensen, M. Stengel, R. Hollmann, R.G. Grainger, 2018: The Community Cloud retrieval for Climate (CC4Cl) – Part 2: The optimal estimation approach. Atmos. Meas. Tech., 11. 3397-3431, DOI: 10.5194/amt-11-3397-2018.

Rodgers, C.D., 2000: Inverse methods for atmospheric sounding, World Scientific Publishing Pte Ltd, New York, USA.

Sus, O., M. Stengel, S. Stapelberg, G. McGarragh, C.A. Poulsen, A.C. Povey, C. Schlundt, G.E. Thomas, M. Christensen, S. Proud, M. Jerg, R.G. Grainger, R. Hollmann, 2018: The Community Cloud retrieval for Climate (CC4Cl) – Part 1: A framework applied to multiple satellite imaging sensors. Atmos. Meas. Tech., 11, 3373-3396, DOI: 10.5194/amt-11-3373-2018.

Related articles

• Page:
  CP: Climate Change Initiative (CCI)
• Page:
  SRB CCI-ICDR: System Quality Assurance Document (SQAD)
• Page:
  SRB CCI-ICDR: Product Quality Assessment Report (PQAR)
• Page:
  SRB CCI-ICDR: Product Quality Assurance Document (PQAD)
• Page:
  SRB CCI-ICDR: Product User Guide and Specification (PUGS)