Contributors: G.E. Thomas (UKRI-STFC RAL Space)

Issued by: STFC RAL Space (UKRI-STFC) / Gareth Thomas

Date: 26/01/2023

Ref: C3S2_D312a_Lot1.2.3.3-v4.0_202301_ATBD_CCISurfaceRadiationBudget_v1.2

Official reference number service contract: 2021/C3S2_312a_Lot1_DWD/SC1

Table of Contents

History of modifications

Version

Date

Description of modification

Chapters/Sections

V1.0

28/07/2022

Brought forward from previous phase of C3S. Updated to include mention of merged Sentinel-3A and -3B level-3 products

All

V1.1

14/12/2022

Updated based on review comments

All

V1.2

26/01/2023

Updated based on review comments

All

List of datasets covered by this document

Deliverable ID

Product title

Product type (CDR, ICDR)

Version number

Delivery date

D3.3.8-v3.0

ECV Surface Radiation Budget brokered from ESA’s Cloud_cci ATSR-AATSRv3 dataset

CDR

V3.0

30/04/2020

D3.3.7-v3.x

ECV Surface Radiation Budget derived from SLSTR

ICDR

V3.1

30/11/2020 - 30/09/2021

D2.1.1-P1/2
D2.1.3-P1

ECV Surface Radiation Budget derived from SLSTR

ICDR

V3.1.1
V4.0

31/05/2022 - onward

Related documents

Reference ID

Document

D1

Algorithm Theoretical Basis Document, v.6.2. ESA Cloud_cci.

https://climate.esa.int/media/documents/Cloud_Algorithm-Theoretical-Baseline-Document-ATBD_v6.2.pdf  

Last accessed on 16/05/2023

D2

Product User Guide, v5.1. ESA Cloud_cci,

https://climate.esa.int/media/documents/Cloud_Product-User-Guide-PUG_v5.1.pdf  

Last accessed on 16/05/2023

D3

Comprehensive Error Characterization Report, v.4.1. ESA Cloud_cci.

https://climate.esa.int/media/documents/Cloud_Comprehensive-Error-Characterisation-Report-CECR_v4.1.pdf

Last accessed on 16/05/2023

D4

Algorithm Theoretical Basis Document: CC4Cl Broadband Radiative Flux Retrieval, v1.1. ESA Cloud_cci,

https://climate.esa.int/media/documents/Cloud_Algorithm-Theoretical-Baseline-Document-ATBD-CC4CL-TOAFLUX_v1.1.pdf

Last accessed on 16/05/2023

D5

Algorithm Theoretical Basis Document (AATSR Oxford-RAL Aerosol and Cloud), Version 3.0. ESA Aerosol_cci.

https://climate.esa.int/sites/default/files/Aerosol_cci2_ATBD_ATSR_ORAC_v3.0.pdf

Last accessed on 16/05/2023

D6

Thomas, G. (2023) C3S Cloud Properties

Service: Algorithm Theoretical Basis Document. Copernicus Climate Change Service,

Document ref. C3S2_D312a_Lot1.2.1.3-v4.0_202301_ATBD_CCICloudProperties_v1.2

CP CCI ICDR: Algorithm Theoretical Basis Document

Last accessed on 16/05/2023

Acronyms

Acronym

Definition

AATSR

Advanced Along-Track Scanning Radiometer

(A)ATSR

Refers to both ATSR-2 and AATSR instruments.

ATBD

Algorithm Theoretical Basis Document

ATSR-1, -2

Along track scanning radiometer

AVHRR

Advanced Very-High-Resolution Radiometer

BRFR

Broadband Radiative Flux Retrieval

C3S

Copernicus Climate Change Service

CC4CL

Community Cloud retrieval for Climate

CCI

(ESA) Climate Change Initiative

CDR

Climate Data Record

CDS

Climate Data Store

CF

Climate-Forecast (model metadata conventions)

DARD

Data Access Requirements Document

ECMWF

European Centre for Medium-range Weather Forecasts

ECV

Essential Climate Variable

ERA

ECMWF Re-Analysis

ERS-2

Environmental Research Satellite

ENVISAT

(the) Environmental Satellite

ESA

European Space Agency

FCDR

Fundamental Climate Data Record

GCMD

Global Change Master Directory

ICDR

Interim Climate Data Record

IREMIS

Infrared Emissivity

MODIS

Moderate Resolution Imaging Spectroradiometer

MWIR

Mid-Wavelength Infrared

NASA

National Aeronautics and Space Administration

NEΔT

Noise-Equivalent delta-Temperature

NOAA

National Oceanic and Atmospheric Administration

OE

Optimal Estimation

ORAC

Optimal Retrieval of Aerosol and Cloud

PUG

Product User Guide

RAL

Rutherford Appleton Laboratory

RITS

Radiative Important Trace Species

RTTOVS

Radiative Transfer for the TIROS Operational Vertical Sounder

SLSTR

Sea and Land surface Temperature radiometer

SNR

Signal-to-Noise Ratio

SOHO

Solar and Heliospheric Observatory

SORCE

SOlar Radiation and Climate Experiment

STFC

Science and Technology Facilities Council

SWIR

Short-Wave Infrared

TCDR

Thematic Climate Data Record

TIR

Thermal Infrared

TIROS

Television Infrared Observational Satellites

TOA

Top-Of-Atmosphere

TSI

Total Solar Irradiance

UKRI

UK Research and Innovation

VIS

Visible (wavelengths)

List of tables

Table 1-1: The specifications of the (A)ATSR (i.e. ATSR-2 and AATSR) instruments

Table 1-2: The orbital characteristics of the ERS-2, ENVISAT and Sentinel-3 satellites

Table 1-3: The specifications of the SLSTR instruments (taken from the SLSTR User Guide (see References))

Table 2-1: (A)ATSR and SLSTR channels used to produce the Cloud_cci Surface Radiation Budget v3.0 Thematic-CDR (TCDR ) and SLSTR v3.x and v4.0 ICDR

Table 2-2: Auxiliary data used in generating the Cloud_cci TCDR and SLSTR ICDR Surface Radiation Budget products

Table 3-1: Inputs to the BRFR module. Cloud properties are only required where the cloud-type is not set to clear. Conversely, aerosol optical depth is only utilized for clear pixels

Table 4-1: The data variables included in the Cloud_cci TCDR and SLSTR ICDR Surface Radiation Budget files brokered to, or produced for, the CDS

Table 4-2: List of global attributes included in the Cloud_cci TCDR and SLSTR ICDR Surface Radiation Budget files brokered to, or produced for, the CDS

List of figures

Figure 1-1: (A)ATSR dual-view scanning system

Figure 1-2: Time coverage of ATSR-2, AATSR and SLSTR at the time of writing

General definitions

Table 1: Summary of variables and definitions

Variables

Abbreviation

Definition

Surface incoming solar radiation

SIS

The total incoming solar flux, measured at the Earth’s surface.

Surface reflected solar radiation

SRS

The total upwelling shortwave flux, measured at the Earth’s surface.

Surface net solar radiation

SNS

The net downwelling solar flux, measured at the surface (equal to SIS – SRS).

Surface downwelling longwave radiation

SDL

 

The total downwelling thermal infrared flux, measured at the Earth’s surface.

Surface outgoing longwave radiation

 

SOL

 

The total upwelling thermal infrared flux, measured at the Earth’s surface.

Surface net longwave radiation

SNL

The net downwelling thermal infrared flux, measured at the Earth’s surface (equal to SDL-SOL).

Total surface radiation budget

SRB

The total net downwelling radiative flux, measured at the Earth’s surface (equal to (SIS+SDL) – (SRS+SOL)).

Table 2: Definition of processing levels

Processing level

Definition

Level-1b

The full-resolution geolocated radiometric measurements (for each view and each channel), rebinned onto a regular spatial grid.

Level-2 (L2)

Retrieved cloud variables at full input data resolution, thus with the same resolution and location as the sensor measurements (Level-1b).

Level-3C (L3C)

Cloud properties of Level-2 orbits of one single sensor combined (averaged) on a global spatial grid. Both daily and monthly products provided through C3S are Level-3C.

Table 3: Definition of various technical terms used in the document

Jargon

Definition

Brokered product

The C3S Climate Data Store (CDS) provides both data produced specifically for C3S and so-called brokered products. The latter are existing products produced under an independent programme or project which are made available through the CDS.

Climate Data Store (CDS)

The front-end and delivery mechanism for data made available through C3S.

Near-real-time (NRT)

Data which is provided within a short time window (often taken to be three hours, but there is no fixed definition) of the measurement. NRT data is often supplanted by a subsequent data stream, which is subjected to more rigorous  data quality checks.

BUGSrad

The broad-band radiative transfer software used to generate the radiative flux values that make up the SRB CDR.

Radiative transfer

The mathematical modelling of the interaction of electromagnetic radiation with some medium – in this case solar and thermal-infrared radiation passing through the Earth’s atmosphere.

Retrieval

A numerical data analysis scheme which uses some form of mathematical inversion to derive physical properties from some form of measurement. In this case, the derivation of cloud properties from satellite measured radiances.

Forward model

A deterministic model which predicts the measurements made of a system, given its physical properties. The forward model is the function which is mathematically inverted by a retrieval scheme. In this case, the forward model predicts the radiances measured by a satellite instrument as a function of atmospheric and surface state, and cloud properties.

TCDR

It is a consistently-processed time series of a geophysical variable of sufficient length and quality.

ICDR

An Interim Climate Data Record (ICDR) denotes an extension of TCDR, processed with a processing system as consistent as possible to the generation of TCDR.

CDR

A Climate Data Record (CDR) is defined as a time series of measurements with sufficient length, consistency, and continuity to determine climate variability and change.

Scope of the document

This Algorithm Theoretical Basis Document (ATBD) is Associated with the CDS catalogue entry: Surface radiation budget from 1982 to present derived from satellite observations. The ATBD describes the algorithm used to generate the Climate Data Record (CDR) on Surface Radiation Budget, which is a brokered service from ESAs Cloud_cci, and its extension with an interim-CDR (ICDR) derived from the Sea and Land Surface Temperature Radiometer (SLSTR).

This extension of is generated specifically for the Copernicus Climate Change Service (C3S) by RAL Space, following the same algorithm and processing chain developed under the ESA’s Cloud cci. Therefore, this document refers to several Cloud_cci documents (see Related documents), namely the Cloud_cci ATBD [D1], the Broadband Radiative Flux Retrieval ATBD [D4] and the ESA Cloud_cci Product User Guide (PUG) [D2]. These documents describe the data processing chain and the algorithms used to generate the CDR products and the cloud products on which they are based. The assessment described in this document is carried out within the scope of C3S, whereas the intellectual property rights of the products themselves remain with the Cloud_cci, in the case of the CDR, or lie with STFC RAL Space, in the case of the ICDR. This document is not part of the official Cloud_cci documentation, but produced solely in the scope of the brokering to the CDS.

Executive summary

The CDR on Surface Radiation Budget is a brokered product of the Community Cloud retrieval for Climate (CC4CL) dataset produced by ESA’s Cloud_cci project, and the SLSTR ICDR is an extension to the brokered product.

The Cloud_cci record contains 17 years (1995-2012) of satellite-borne observations derived from measurements of the Along Track Scanning Radiometer (ATSR) series of satellites on board the ESA Environmental Research Satellite (ERS-2) and ENVISAT satellites. The CDR provided to C3S comprises monthly means (0.5° x 0.5° resolution) of Surface Radiation Budget on a regular global latitude-longitude grid. The SLSTR ICDR continues this record, with a 5-year gap, with version v3.x providing Surface Radiation Budget products from Sentinel-3A SLSTR from 2017 onwards, and from Sentinel-3B from late 2018 onwards. Surface radiation products are provided individually from each Sentinel-3 platform, and as a combined  product that averages data from both SLSTR instruments into single daily and monthly means.

The Surface Radiation Budget products are surface incoming and reflected shortwave (solar) radiation, surface downwelling and outgoing longwave (thermal-IR) radiation, net-fluxes of shortwave and longwave radiation and total surface radiation budget (shortwave + longwave). Note that the service within Copernicus provides a set of additional products to those provided by the original Cloud_cci products: only the relevant Cloud_cci products are included in the product brokered to the Climate Data Store (CDS) and the net fluxes, providing the Surface Radiation Budget ECVs, are calculated directly from the downwelling and upwelling fluxes.

This document is divided into the following sections:

1. Instruments

Describes the characteristics of the instruments and satellite platforms used to produce the data which make up the primary input to the cloud-properties processing chain, or provides references for this information. This amounts to a general description of the measurements used as the primary inputs for the analysis, including the spectral bands used, spatial and temporal resolution and coverage.

2. Input and auxiliary data

This section specifies the particular data products used as inputs to the cloud-properties CDR processing chain, or provides references for this information. This includes both the primary input “fundamental climate data record” and the various ancillary data also required by the processing chain.

3. Algorithms

Provides references which describe the processing chain and algorithms used to produce the CDR. These include the retrieval scheme, which derives level-2 cloud-properties from the top-of-atmosphere radiances measured by satellite instruments, and the level-3 processing, which averages the level-2 data onto defined latitude-longitude grids to produce the daily and monthly products provided by C3S. A description of uncertainty characterization is also included in this section.

4. Output data

Describes the format and content of the output files which constitute the cloud-properties CDR provided by C3S.

1. Instruments

The brokered Cloud_cci Surface Radiation Budget dataset is derived from the ATSR-2 and Advanced Along-Track Scanning Radiometer (AATSR) instruments, which are described in Cloud_cci ATBD [D1], Section 2.2. ATSR-2 flew on board the second European Research Satellite (ERS-2), which provide global data coverage from mid-1995 until mid-2003, while AATSR flew on ENVISAT from mid-2002 until April 2012 (see Figure 1‑2). Both satellites were in Sun-synchronous polar orbits, with ENVISAT following the same ground-track as ERS-2, but with an overpass time 30 minutes ahead. A summary of the orbital characteristics of ERS-2 and ENVISAT are given in Table 1-2. ATSR-2 and AATSR were essentially identical instruments, with the same set of channels providing the same 1×1 km nominal spatial resolution and 512 km wide swath. Orbital characteristics of (A)ATSR are summarised in Table 1-1. The unique feature of the ATSR instruments was their dual-view measurement system (see Figure 1-1). The instruments used a spinning mirror to produce a conical scan pattern, in which the instrument views both directly down (the nadir-view) and in the direction of the satellite’s orbital path, centred at a zenith angle of 55° (the forward-view). The dual-view is key to the accuracy of the ATSR instruments in their primary role of measuring sea surface temperature and is also very useful for other applications, such as making measurements of atmospheric aerosol. However, for the determination of cloud properties, only the nadir-view is used. Another key feature of the ATSR instruments was their high accuracy. During every sweep of the conical scan pattern, the instrument views two on-board temperature controlled back-body targets, which provide on-going calibration of the thermal channels. Additionally, the instrument views the sun through an opal filter once per-orbit, which provides shortwave channel calibration.

Figure 1‑1: (A)ATSR dual-view scanning system.

One significant difference between the two instruments is that ERS-2 did not provide enough communication bandwidth for all ATSR-2 data to be fully downlinked. This limitation is dealt with by two data-saving data modes:

  1. The so-called “narrow-swath” mode, whereby only the centre 256 pixels of the swath are available for 1 or more shortwave channels. This mode was only active over the ocean and the 550 nm channel-1 is the most often effected.
  2. Additionally, the number of bits used to encode the radiance measured by some shortwave channels is sometimes lowered to 8 from the nominal 12. Again, this only occurs over the ocean.

Table 1-1: The specifications of the (A)ATSR (i.e. ATSR-2 and AATSR) instruments.

 

Capability

(A)ATSR Specifications

Swath

Nadir view

512 km

Oblique view

512 km

Global Coverage Revisit Times


3-4 day (mean)

Spatial Sampling interval at Sub-satellite point (km)

VIS-SWIR

1 km

IR

1 km

Spectral channel centre (µm)

VIS

0.554 (Ch1); 0.659 (Ch2) 0.865 (Ch3)

SWIR

1.61 (Ch4)

MWIR/TIR

3.70 (Ch5); 10.85 (Ch6); 12.00 (Ch7)

Figure 12: Time coverage of ATSR-2, AATSR and SLSTR at the time of writing.

ICDR data is derived from the Sea and Land Surface Temperature Radiometer (SLSTR), flown on board the Copernicus Sentinel-3 platform. The SLSTR is an improved version of the ATSR instruments, providing additional spectral channels, improved spatial resolution for the visible and shortwave-infrared channels and a considerably wider swath (1,400 km nadir swath, compared to 512 km for AATSR). Otherwise, SLSTR maintains the main features of the ATSR series; namely the dual-view and on-board calibration systems. As part of the Copernicus operational observation system, two SLSTR instruments are kept operational, with a backup instrument also to be in orbit (although not yet launched). The two operational instruments were launched on Sentinel-3A on 16th February 2016 and Sentinel-3B on 25th April 2018. The two platforms fly in identical, interleaved sun-synchronous orbits, such that the two SLSTR instruments provide nearly global coverage twice daily (with one day and one night overpass). Orbital characteristics of Sentinel-3A are summarised in Table 1-3.

Table 1-2: The orbital characteristics of the ERS-2, ENVISAT and Sentinel-3 satellites.

Platform

Altitude (km)

Inclination

Period (min)

Repeat Cycle (days)

Ground-track deviation

Local Time at Descending node

ERS-2

780

98.5°

100

35


10:30

ENVISAT

799.8

98.55°

100.59

35


10:00

Sentinel-3

814.5

98.65°

100.99

27

±1 km

10:00

The SLSTR instrument is described in detail by the Sentinel-3 SLSTR User Guide (see References), but an overview of its specifications is given in Table 1-3.

Table 1-3: The specifications of the SLSTR instruments (taken from the SLSTR User Guide (see References)).

 

Capability

SLSTR Specifications

Swath

Nadir view

1,400 km

Oblique view

740 km

Global Coverage Revisit Times

(nadir view)

1 satellite


1 day (mean)

2 satellites

0.5 day (mean)

Spatial Sampling interval at Sub-satellite point (km)

VIS-SWIR

0.5 km

IR-Fire

1 km

Spectral channel centre (µm)

VIS

0.554 (S1); 0.659 (S2) 0.868 (S3)

SWIR

1.374 (S4); 1.613 (S5); 2.25 (S6)

MWIR/TIR

3.742 (S7); 10.85 (S8); 12.02 (S9)

Fire ½

3.742 (F1); 10.85 (F2)

Radiometric Resolution

VIS (Albedo =0.5%)

Signal-to-Noise Ratio (SNR) > 20

SWIR (Albedo =0.5%)

Signal-to-Noise Ratio (SNR) > 20

MWIR (T =270K)

NEΔT < 80 mK

TIR (T=270K)

NEΔT < 50 mK

Fire 1 (<500 K)

NEΔT < 1 K

Fire 2 (<400 K)

NEΔT < 0.5 K

Radiometric Accuracy

VIS-SWIR (Albedo = 2-100%)

< 2% (Beginning of Life)

<5% (End of Life)

MWIR –TIR
(265 – 310 K)

< 0.2 K (0.1 K goal)

Fire (< 500 K)

< 3 K

2. Input and auxiliary data

This section summarises the required input data used in the retrieval algorithms.

2.1 Fundamental climate data record

The input data source for the Cloud_cci dataset is the so-called AATSR-multi-mission level 1b data record, and this forms the fundamental climate data record for this product. The processing of measured radiances to level 1b is described in the ESA AATSR Detailed Processing Model Level 1b (see References) and ENVISAT-style products for ATSR-1 and ATSR-2 data (see References) documents.

The input data source for the SLSTR ICDR is the (non near-real-time) ESA Observation mode SLSTR level-1 archive, hosted by the Centre for Environmental Data Analysis (CEDA). At time of writing, no complete reprocessing of the SLSTR level-1 archive has been undertaken, and improvements in instrument calibration, pixel colocation/geolocation and minor product details have continued to evolve throughout the lifetime of the mission. It is likely that some variation in cloud product quality will be attributable to this inconsistency in the level-1 input data. Details of the quality of SLSTR data can be found in the “Data Product Quality Reports” section of the SLSTR User Guide (see References).

SLSTR level-1 data is provided in NetCDF-4 formatted files, which are segregated into 3-minute frames. The level-1 data provide:

  • Top of Atmosphere (TOA reflectance) (for visible and Short Wave InfraRed (SWIR) channels)
  • Brightness temperature (for Thermal InfraRed (TIR) channels)
  • Pixel viewing geometry
  • Measurement time
  • Geolocation of each pixel

as well as other auxiliary information (see the SLSTR User Guide (see References) for further details).

The radiative flux products (both Surface and Earth Radiation Budget ECVs) produced by CC4CL are generated as a post-processing step after the retrieval of cloud and aerosol properties from the Optimal Retrieval of Aerosol and Cloud (ORAC) retrieval scheme. This post-processor is known as the Broadband Radiative Flux Retrieval (BRFR) module and operates on level 2 cloud properties and aerosol optical depth products generated under the Cloud_cci, Aerosol_cci and C3S initiatives. Details of the Aerosol retrieval scheme employed for Aerosol_cci and C3S can be found in the Aerosol_cci ORAC ATBD [D5].

2.2 Specific input and auxiliary data

The specific spectral channels used in producing the (A)ASTR and SLSTR Surface Radiation Budget datasets are detailed in Table 2‑1, which were chosen to match the so-called heritage-channels provided by the long-running Advanced Very High Resolution Radiometer (AVHRR) instrument series1. The cloud retrieval only makes use of the nadir view of the (A)ATSR and SLSTR instruments.

Table 2-1: (A)ATSR and SLSTR channels used to produce the Cloud_cci Surface Radiation Budget v3.0 Thematic-CDR (TCDR ) and SLSTR v3.x and v4.0 ICDR

(A)ATSR Channel number

SLSTR Band name

Nominal wavelength

2

S2

0.67 μm

3

S3

0.87 μm

4

S5

1.6 μm

6

S8

10.8 μm

7

S9

12.0 μm

In addition to (A)ATSR or SLSTR level 1 data, the CC4CL retrieval scheme also relies on a range of auxiliary datasets, which are detailed in Table 2‑2.

Table 2-2: Auxiliary data used in generating the Cloud_cci TCDR and SLSTR ICDR Surface Radiation Budget products.

Dataset

Description

ECMWF ERA-Interim

ECMWF reanalysis products provide pressure, temperature, humidity and ozone profiles, as well a priori surface temperature, sea ice extent and near-surface wind speed for ocean surface reflectance calculation.

MODIS MCD43A12 V006

The MODIS BRDF product provides land-surface reflectance.

IREMIS UW Baseline Fit

The Global Infrared Land Surface Emissivity (IREMIS) University of Wisconsin-Madison Baseline Fit to the MODIS MOD11 emissivity product provides land-surface emissivity.

RTTOV

The standard coefficient and database files provided with RTTOV v 12.1 are used, where not superseded by other auxiliary data (as is the case with the emissivity and surface reflectance atlases)

SOHO and SORCE

Up until September 2019, information on incoming total solar irradiance (TSI) is supplied by the Solar and Heliospheric Observatory (SOHO) and SOlar Radiation and Climate Experiment (SORCE) satellite records.

NOAA daily TSI CDR and TSIS

From September 2019 on, information on incoming total solar irradiance is supplied by the Total Solar Irradiance CDR produced by NOAA, with measurements from the Total and Spectral Solar Irradiance Sensor (TSIS) on the International Space Station being used for dates too recent to be covered by the NOAA CDR.

Further details of the input and auxiliary data used are provided in the Cloud_cci CC4CL ATBD [D4], Section 3. Full details of input data to the BRFR module are given in Section 2.1 of the BRFR ATBD [D4].

1 A key objective of the cloud_cci project under which the (A)ATSR CDR was produced was the production of consistent records of cloud properties from a range of satellite sensors that provided similar measurements (including AVHRR, (A)ATSR and the Moderate Resolution Imaging Spectroradiometer (MODIS)). In order to achieve this, a set of common channels, shared by all included instruments, was used in retrieving cloud properties – these correspond to the channels provided by AVHRR.

2 The product used is MCD43A1 V006. See the MODIS user guide at https://www.umb.edu/spectralmass/terra_aqua_modis/v006 (accessed 7-May-2020).

3. Algorithms

This section describes the algorithms used to derive the final Surface Radiation Budget products. The overall processing scheme used to produce the CDR products, is known as the “Community Cloud for Climate” scheme or CC4CL, includes several processing steps:

  • Pre-processing, which involves ingestion of input and auxiliary data, cloud masking, and performing clear-sky radiative transfer using the RTTOV package.
  • Cloud-properties retrieval on pixels identified as cloud during pre-processing, using the Optimal Retrieval of Aerosol and Cloud (ORAC) algorithm. For (A)ATSR and SLSTR processing this step is run twice, once assuming the presence of liquid-water cloud and once assuming ice cloud.
  • The combination of the two separate ORAC outputs into a single level-2 cloud product, which each pixel defined as either clear-sky, liquid-water cloud or ice cloud (noting that, in this context, clear-sky means any pixel determined not to contain water cloud).
  • The generation of radiative-fluxes, based on the level-2 cloud product and auxiliary aerosol and TSI data, using the BRFR module.
  • The compositing of the level-2 products into level-3c, containing daily or monthly averaged cloud properties on a regular latitude-longitude grid.

Each of these steps will be briefly summarized below, with references to existing documents which describe the scheme in detail.

Note that the scheme applied to the (A)ATSR CDR and SLSTR ICDR products is virtually identical. The only differences relate to the mechanics of reading the different ENVISAT and Copernicus style level-1b products, and handling the different specific instrument and platform names.

3.1 Pre-processing

The ORAC processor is designed as a general-purpose cloud and aerosol retrieval scheme, which can be applied relatively straight-forwardly to a wide range of visible-infra satellite imaging instruments. To facilitate this flexibility, ORAC is coupled with a pre-processor, which deals with the differences in instrument characteristics, data formats and auxiliary data and presents the retrieval scheme with a set of standard input files. The details of the pre-processing can be found in the Cloud_cci ATBD [D1], Sections 2-3 and the CC4CL ATBD [D4]. Here we provide an outline of the steps involved:

  1. The pre-processor is provided with a level-1b product of a support instrument. This data is read in, from which the location and time of measurement is determined, in addition to the radiance, reflectance and/or brightness temperature data, viewing geometry and other information required by the retrieval scheme.
  2. Ancillary data is then read in for the correct location and time. This includes reanalysis data for the atmospheric state (ECMWF ERA5 in this case) and data on surface properties, including land-surface emissivity and reflectance, ocean colour and ice/snow cover. In the case of land emissivity, reflectance and ocean colour, climatological data are used if data is unavailable for the specific time and location.

The atmospheric state, surface albedo and surface temperature data also form inputs for the BRFR module directly.

  1. Cloud detection and masking is performed. This set is described in more detail in Section 1.1.1.
  2. The surface bidirectional reflectance, along with hemispherical and bi-hemispherical reflectance, is estimated for each level-1b pixel. This is based on the ancillary surface reflectance data for land. Over water an ocean surface reflectance model, which is driven by near-surface wind vectors from reanalysis data combined with ocean colour data is used.
  3. Clear-sky (i.e. representing the atmosphere in the absence of cloud) radiative transfer is calculated by RTTOV for the area covered by the level-1b data, using the reanalysis data as input. This produces estimates of atmospheric transmission and emissivity above and below each layer within the atmospheric profile provided by the reanalysis, for each instrument channel and at the viewing geometry defined by the level-1b data.
  4. The level-1b radiances, processed ancillary data and RTTOV output are output as a standard set of NetCDF format files, which are independent of the instrument being used and the source of ancillary data, and form the input of the retrieval process itself.

3.2 Retrieval of swath-based surface radiation (level-2 data)

3.2.1 Cloud fractional cover

The algorithm used to retrieve the level-2 data on the cloud fractional cover is briefly summarised in  Cloud_cci ATBD [D1] Section 3.1 and the CC4CL ATBD [D4] Section 2.1. A comprehensive description can be found in Sus et al. 2017.

Cloud masking is performed on level-1b radiances and is based on an artificial neural network (ANN) that has been trained using AVHRR-NOAA-19 measurements and collocated with CALIOP. As most satellite instruments do not provide suitable colocations with CALIOP due to orbital differences (AATSR and SLSTR are only offer colocations in a small band at high-latitudes, for instance), the cloud masking includes a step of adjusting the radiances observed by the target instrument in the heritage channels to best simulate the spectral response of AVHRR-NOAA-19. The coefficients used to perform this adjustment are calculated by convolving the heritage channel spectral response functions of the target instrument and NOAA-19.

The level-2 cloud mask is used to determine which level-1 pixels have the cloud properties retrieval algorithm applied and are used to derive the total cloud fractional coverage in the daily and monthly level-3c CDR products.

3.2.2 Cloud physical properties

The Optimal Retrieval of Aerosol and Cloud (ORAC) algorithm used to derive level-2 cloud properties (cloud optical-depth, effective radius, cloud-top pressure and derived properties) is summarised in the C3S ATBD for the Cloud_cci cloud properties CDR [D6], with further details provided in the Section 3.3 of the Cloud_cci ATBD [D1], the CC4CL ATBD [D4] and McGarragh et al 2017.

ORAC is an optimal estimation (OE) algorithm, which fits a parameterised model of top-of-atmosphere radiance and brightness temperature (known as the forward model) to satellite observations in a number of channels. The forward model consists of a simple representation of a cloud layer within the clear-sky atmosphere modelled by RTTOV in the pre-processor. The transmission, reflectance and emissivity of the cloud layer for each instrument channel is stored in look-up tables which are a function of solar and viewing geometry and cloud optical depth and effective radius. This model is fitted to the satellite observations, such that the deviation of the modelled values from the observations are minimised, by varying cloud optical depth, effective radius and the position of the cloud layer within the RTTOV modelled atmosphere. The use of OE means that all retrieval parameters are retrieved simultaneously as a function of all measurement channels. This is a key feature in using the derived cloud properties as inputs to the radiative transfer calculations performed by the BRFR module, as it ensures they are radiative consistent from the shortwave through to the thermal infrared3.

3This contrasts with algorithms which retrieve optical depth and effective radius using shortwave channels alone, while cloud-top height is derived solely from the thermal window channels.

3.3 Retrieval of swath-based Surface Radiation Budget (level-2 data)

The algorithm used to derive surface (and top-of-atmosphere) radiation is described in Section 2.1 of the Cloud_cci BRFR ATBD [D4].

The BRFR is built around the BUGSrad radiative transfer code, which itself is an implementation of the two-stream approximation and correlated-k distribution methods of atmospheric radiative transfer (see Fu and Liou 1992 and Stephens et al. 2001 for details of these methods and their implementation in BUGSrad). The BUGSrad software itself is in the public domain and may be obtained through its own website (see References).

BUGSrad, is a 1-D radiative transfer (i.e. it treats the medium through which radiation  passes as homogenous, except in the direction along which transmission/absorption is being calculated). It uses the correlated-k approximation, where the spectral dependence of trace-gas absorption is reduced to coefficients for broad spectral bands, to derive the atmospheric absorption across 18 bands covering the range from 200 nm through to the far infrared (the last band simulating all infrared wavelengths above 36 µm). Propagation of radiation through the atmosphere is modelled using the two-stream approach, where only up- and down-welling irradiance are explicitly calculated at each vertical level.

The primary inputs to the BRDR module are listed in Table 3-1. With the exception of the TSI and aerosol optical depth, all of these quantities are provided by the CC4CL pre-processor and ORAC cloud retrieval. Aerosol optical depth is an optional input parameter, with a fixed value of 0.05 being assumed if it is not supplied. For the Cloud_cci TCDR of surface radiation budget, level-2 Aerosol_cci data (also derived from (A)ATSR using the ORAC algorithm) was used. Aerosol optical depth was not included as an input parameter, due to a lack of appropriate data4.

Table 3-1: Inputs to the BRFR module. Cloud properties are only required where the cloud-type is not set to clear. Conversely, aerosol optical depth is only utilized for clear pixels.

BUGSrad input

Data source

Total solar irradiance (TSI)

SOHO and SORCE, or

NOAA daily TSI CDR and TSIS

Solar zenith angle

ORAC output (from L1 data)

Direct and diffuse albedo of the surface (visible and near-IR)

Pre-processor output

Surface temperature

Pre-processor output (ERA5)

Cloud type (clear-sky, liquid water cloud or ice cloud)

ORAC output

Cloud-top height

ORAC output

Cloud optical depth (at 550 nm)

ORAC output

Cloud effective radius

ORAC output

Profiles of temperature, pressure, humidity and ozone-concentration, as a function of height.

Pre-processor output (ERA5)

Aerosol optical depth (optional)

Aerosol_cci L2, C3S aerosol L2 or MODIS L2.

4 C3S does provide SLSTR aerosol products, including one derived using ORAC, but the difference in nadir and oblique swath widths in SLSTR means that aerosol properties are not available for the full swath of the cloud retrievals.

3.4 Generation of final products (level-3 data)

The generation of the final level-3 products (monthly means) is outlined in Cloud_cci ATBD [D1], Section 4.

The generation of monthly averaged radiative flux estimates is considerably more complex than for many satellite products, where the usual approach is to simply calculate the mean of all data within each cell of a fixed spatial grid across the month. Such an approach applied to radiative fluxes derived from a sun-synchronous satellite would certainly provide an estimate of the radiation budget at the specific local solar time of the satellite overpass, but, due to the dominant role of the diurnal cycle on atmospheric radiative fluxes throughout the day, such an average would not be directly relatable to the true radiation budget across the month. Thus, the diurnal cycle of solar irradiance, as a function of date and latitude, is applied to the level-2 observations (for the daylight overpass of the satellite), to produce a true daily average under the assumption that the atmospheric state at the time of observation is representative of the conditions across the whole day. Details of this approach are given in the Cloud_cci ATBD [D1], Section 4.

A set of additional level-3 files have been generated specifically for the SLSTR ICDR, combining additional data from instruments on both Sentinel-3A and -3B. These files are functionally identical to those produced for the individual SLSTR instruments, with the same structure, variables and input level-2 data, except that they combine data from both instruments, rather than just one. As the two Sentinel-3 platforms are in interleaved sun-synchronous orbits, combining data from both SLSTR instruments provides near global coverage twice-daily. The level-3 algorithm used to produce these merged files is identical to that used produce level-3 from each instrument separately.

3.5 Additional CDS radiative properties

As mentioned above the products brokered to the CDS contain a different set of properties than those included in the original Cloud_cci products. In particular, the CDS products include net-flux (Surface Radiation Budget) values. These are calculated directly from the retrieved up- and down- welling fluxes (and thus do not constitute additional information). The surface net shortwave radiation (sns) is calculated using:

$$sns=sis-srs \ \ (Eq. 1)$$

where sis and srs are the incoming and reflected surface shortwave radiation respectively. Likewise, the surface net longwave radiation (snl) is calculated from:

$$snl=sdl-sol \ \ (Eq. 2)$$

where sdl and sol are the downwelling and outgoing longwave radiation respectively. The total surface radiation budget (srb) is simply the sum of the short- and longwave contributions:

$$srb=(sis+sdl)-(srs+sol) \ \ (Eq. 3)$$

4. Output data

This section summarises information on the output files.

4.1 File format

Cloud_cci TCDR and SLSTR ICDR products are provided to the CDS in NetCDF (version 4), which are compliant with the conventions Climate and Forecast metadata (CF) 1.8 and the NASA Global Change Master Directory (GCMD) Science Keywords vocabulary. Filenames follow the structure:

C3S-312bL1-L3C-MONTHLY-SRB-INST_ORAC_PLATFORM_YYYYMM_fvV.V.nc,

where INST and PLATFORM refer to the instrument and platform from which data originates (either ATSR2 and ERS2, or AATSR and ENVISAT for TCDR data, or SLSTR and Sentinel-3a, -3b or -3a+b for ICDR data for each individual Sentinel-3 platform, or the combined product from both platforms). YYYYMM provides the year and month covered by the monthly mean product, while V.V indicates the product version number (v3.0 for the TCDR, v3.1.1 for the individual Sentinel-3 ICDR and v4.0 for the combined Sentinel-3 product).

4.2 File contents

Table 4-1 lists the parameters included in the output files generated under the C3S for the SRB products. Data are provided as monthly means of Surface Radiation Budget, as described in Table 4-1, on a regular latitude-longitude grid, with a spacing of 0.5° in both dimensions (thus grid centres lie at -89.75°, -89.25°, -88.75°, …, 89.75° in latitude and -179.75°, -179.25°, -178.75°, …, 179.75° in longitude). Thus the parameters listed in Table 4-1 are presented as 720x360 element arrays, on the grid defined by the longitude (lon, with 720 elements) and latitude (lat, with 360 elements) axes.

In addition, each file contains the global attributes defined in Table 4-2.

Table 4-1: The data variables included in the Cloud_cci TCDR and SLSTR ICDR Surface Radiation Budget files brokered to, or produced for, the CDS.

Property

Unit

Variable name

Comment

Latitude

Degrees North

lat

Values correspond to grid centres (-89.75°, -89.25°, -88.75°, …, 89.75°)

Longitude

Degrees East

lon

Values correspond to grid centres (-179.75°, -179.25°, -178.75°, …, 179.75°)

Cloud pixel count

-

pixel_count

The number of level 2 cloud pixels included in the averaging (NB. Does not include number of Aerosol_cci pixels included).

Surface incoming solar radiation

Wm-2

sis


Standard deviation in sis

Wm-2

sis_std

The spatial-temporal standard deviation in the level 2 sis values over the month and grid-cell, if available.

Surface reflected solar radiation

Wm-2

srs


Standard deviation in srs


srs_std

See sis_std.

Surface net solar radiation

Wm-2

sns

Equivalent to "Surface ERB shortwave" GCOS ECV.

Standard deviation in sns


sns_std

See sis_std.

Surface downwelling longwave radiation

Wm-2

sdl


Standard deviation in sdl


sdl_std

See sis_std.

Surface outgoing longwave radiation

Wm-2

sol


Standard deviation in sol


sol_std

See sis_std.

Surface net longwave radiation

Wm-2

snl

Equivalent to "Surface ERB longwave" GCOS ECV and calculated as snl = sdl – sol.

Standard deviation in snl


snl_std

See sis_std.

Total surface radiation budget

Wm-2

srb

Calculated as srb = (sis + sdl) - (srs + sol).

Standard deviation in srb


srb_std

See sis_std.

Table 4-2: List of global attributes included in the Cloud_cci TCDR and SLSTR ICDR Surface Radiation Budget files brokered to, or produced for, the CDS.

Attribute name

Description

title

Descriptive title of the file contents

project

"Climate Change Initiative-European Space Agency" or "Copernicus Climate Change Service"

product_version

The version number of the product

conventions

Lists the naming and meta data conventions used

standard_name_vocabulary

Defines the standard name convention used

institution

The source institution of the data

source

The source (level 1) data used in the product

geospatial_lon_resolution

The grid spacing in the longitude axis

geospatial_lat_resolution

The grid spacing in the latitude axis

geospatial_lon_min

The minimum longitude covered by the product

geospatial_lon_max

The maximum longitude covered by the product

geospatial_lat_min

The minimum latitude covered by the product

geospatial_lat_max

The maximum latitude covered by the product

spatial_resolution

Alternative description of the spatial grid used

geospatial_vertical_min

Definition of vertical grid used (0 indicated no vertical grid)

geospatial_vertical_max

Definition of vertical grid used (0 indicated no vertical grid)

platform

Satellite platform from which observations originated

sensor

Satellite sensor which made the observations used

creator_email

contact.c3s312b@stfc.ac.uk

creator_url

www.esa-cloud-cci.org

date_created

ISO date and time string of processing time of the particular file

creator_name

Alternative for institution

time_coverage_duration

ISO time string defining temporal coverage of the product

time_coverage_resolution

ISO time string defining temporal resolution of the product

references

Link to further information about the product

history

Brief description of provenance of the product

summary

Brief description of product contents

keywords

List of keywords (for data discovery)

comment

Any further comments on the product not covered by other fields

license

License conditions of the product

cdm_data_type

Common Data Model Data Type used in the NetCDF file

keywords_vocabulary

The standard list from which the keywords have been extracted

naming_authority

ID string of the institution naming product (and contents)

tracking_id

An ISO Universally Unique Identifier (UUID) number for the file

id

A human readable identifier of the product

time_coverage_start

ISO time string of the start of the product's temporal coverage

time_coverage_end

ISO time string of the end of the product's temporal coverage

inputfilelist

List of the primary input files used to create the product

References

AATSR Detailed Processing Model Level 1b. Ref: PO-TN-RAL-GS-10004, https://earth.esa.int/eogateway/documents/20142/37627/AATSR-Detailed-Processing-Model-Level-1B.pdf (Last accessed on 23/01/2023)

BUGSrad: Radiative transfer with gases and clouds. https://biocycle.atmos.colostate.edu/shiny/BUGSrad/ Last accessed on 23/01/2023

ENVISAT-style products for ATSR-1 and ATSR-2 data. Ref: APP-TN-005, https://earth.esa.int/eogateway/documents/20142/37627/Envisat-style%20products%20for%20ATSR-1%20and%20ATSR-2%20data (Last accessed on 23/01/2023)

Fu and Liou. “On the correlated k-distribution method for radiative transfer in nonhomogeneous atmospheres”. J. Atmos. Sci. 1992. DOI: https://doi.org/10.1175/1520-0469(1992)049%3C2139:OTCDMF%3E2.0.CO;2

McGarragh et al. "The Community Cloud retrieval for CLimate (CC4CL). Part II: The optimal estimation approach", Atmospheric Measurement Techniques (2017). DOI: https://doi.org/10.5194/amt-2017-333

Sentinel-3 SLSTR Technical Guide. https://sentinels.copernicus.eu/web/sentinel/technical-guides/sentinel-3-slstr/instrument (Last accecssed on 23/01/2023)

Stephens et al. Parameterization of Atmospheric Radiative Transfer. Part I: Validity of Simple Models, J. Atmos. Sci. 2001. DOI: https://doi.org/10.1175/1520-0469(2001)058%3C3391:POARTP%3E2.0.CO;2

Sus et al. "The Community Cloud retrieval for CLimate (CC4CL)–Part 1: A framework applied to multiple satellite imaging sensors.", Atmospheric Measurement Techniques (2017). DOI: https://doi.org/10.5194/amt-2017-334

References are listed in ESA Cloud CCI ATBD [D1], Section 6; in ESA Cloud CCI PUG [D2], Section 6; in ESA Cloud CCI CECR [D3], Section 11; in ESA Cloud CCI BRFR ATBD [D4], Section 4; in AATSR Level 1b Detailed Processing Model, Section 3; in ENVISAT-style Products for ATSR-1 and ATSR-2 data, Section 8; and Sus et al. and Mcgarragh et al.

This document has been produced in the context of the Copernicus Climate Change Service (C3S).

The activities leading to these results have been contracted by the European Centre for Medium-Range Weather Forecasts, operator of C3S on behalf of the European Union (Delegation Agreement signed on 11/11/2014 and Contribution Agreement signed on 22/07/2021). All information in this document is provided "as is" and no guarantee or warranty is given that the information is fit for any particular purpose.

The users thereof use the information at their sole risk and liability. For the avoidance of all doubt , the European Commission and the European Centre for Medium - Range Weather Forecasts have no liability in respect of this document, which is merely representing the author's view.

Related articles