Contributors: Karl-Göran Karlsson (SMHI), Oleksandr Bobryshev (DWD), Anna-Christina Mikalsen (DWD), Tim Usedly (DWD), Gareth Thomas (UKRI-STFC RAL Space)

Issued by: SMHI/Karl-Göran Karlsson

Date: 03/06/2024

Ref: C3S2_D312a_Lot1.3.1.1-2023_TRGAD-SRB_v1.1

Official reference number service contract: 2021/C3S2_312a_Lot1_DWD/SC1

Please cite as

Karlsson, K.-G., et al., (2024): C3S Surface Radiation Budget CDRs releases until March 2024: Target Requirements and Gap Analysis Document. Copernicus Climate Change Service. Document reference C3S2_D312a_Lot1.3.1.1-2023_TRGAD-SRB_v1.1. Last accessed on dd/mm/yyyy

Table of Contents

History of modifications

Version

Date

Description of modification

Chapters / Sections

V1.0 

30/04/2024

Original version covering all deliveries between start of Phase II until March 2024

All

V1.1

03/06/2024

Document revised following feedback from independent review

All

Related documents

Reference ID

Document

D1

Algorithm Theoretical Basis Document, CM SAF Cloud, Albedo, Radiation data record, AVHRR-based, Edition 3 (CLARA-A3) Surface Radiation
Code: SAF/CM/DWD/ATBD/CLARA/RAD
https://www.cmsaf.eu/SharedDocs/Literatur/document/2023/saf_cm_dwd_atbd_clara_rad_3_3_pdf.pdf?__blob=publicationFile
Last accessed on 07/08/2024

D2

Product User Manual, CM SAF Cloud, Albedo, Radiation data record, AVHRR-based, Edition 3 (CLARA-A3) Surface Radiation
Code: SAF/CM/DWD/PUM/CLARA/RAD
https://www.cmsaf.eu/SharedDocs/Literatur/document/2023/saf_cm_dwd_pum_clara_rad_3_1_pdf.pdf?__blob=publicationFile
Last accessed on 07/08/2024

D3

Validation Report, CM SAF Cloud, Albedo, Radiation data record, AVHRR-based, Edition 3 (CLARA-A3) Surface Radiation
Code: SAF/CM/DWD/VAL/CLARA/RAD, Issue 3.1
https://www.cmsaf.eu/SharedDocs/Literatur/document/2023/saf_cm_dwd_val_clara_rad_3_1_pdf.pdf?__blob=publicationFile
Last accessed on 07/08/2024

D4

CM SAF CDOP2 Product Requirement Document, SAF/CM/DWD/PRD, v4.0

Available upon request from Deutscher Wetterdienst (DWD) 

D5

Algorithm Theoretical Basis Document CM SAF Cloud, Albedo, Radiation data record, AVHRR-based, Edition 3 (CLARA-A3) Cloud Products (level-1 to level-3), Issue 3.3
Code: SAF/CM/DWD/ATBD/CLARA/CLD
https://www.cmsaf.eu/SharedDocs/Literatur/document/2023/saf_cm_dwd_atbd_clara_cld_3_3_pdf.pdf?__blob=publicationFile
Last accessed on 07/08/2024

D6

[GCOS-107] Systematic Observation Requirements for Satellite-based Products for Climate, 2006

https://library.wmo.int/doc_num.php?explnum_id=3813-107

Last accessed on 07/08/2024

D7

[GCOS- 200] Global Climate Observing System, Implementation Plan, 2016. World Meteorological Organization, Geneva, Switzerland.

Available from: https://library.wmo.int/doc_num.php?explnum_id=3417

Last accessed on 07/08/2024

D8

Hollmann R., Mikalsen A.C. (2020), C3S Service: Input - Inventory for each product - 2020, Copernicus Climate Change Service,

Document ref. C3S_D312b_Lot1.3.1.1-2020_Input_Inventory

Not yet in CKB

D9

Meirink, J.F. et al (2024) C3S Service: Key Performance Indicators (KPIs), Copernicus Climate Change Service,

Document ref. C3S_D312b_Lot1.0.4.8_201903_UpdatedKPIs_v1.0 and C3S2_D312a_Lot1.3.7.1_202401_Unified_KPI_Approach_v1.1

https://confluence.ecmwf.int/x/AM_BEQ

Last accessed on 07/08/2024

D10

Algorithm Theoretical Basis Document CM SAF Cloud, Albedo, Radiation data record, AVHRR-based, Edition 3 (CLARA-A3) Surface Black-sky, White-sky and Blue-sky Albedo, Issue: 3.3
Code: SAF/CM/FMI/ATBD/CLARA/SAL

https://www.cmsaf.eu/SharedDocs/Literatur/document/2023/saf_cm_fmi_atbd_clara_sal_3_3_pdf.pdf?__blob=publicationFile

Last accessed on 07/08/2024


D11

Sentinel-3 SLSTR User Guide, ESA , v.5.1, 16.01.2020.

https://climate.esa.int/media/documents/Cloud_Product-User-Guide-PUG_v5.1.pdf

Last accessed on 07/08/2024

D12

ESA Cloud CCI Algorithm Theoretical Basis Document, v.6.2, 14.10.2019.

https://climate.esa.int/media/documents/Cloud_Algorithm-Theoretical-Baseline-Document-ATBD_v6.2.pdf

Last accessed on 07/08/2024

D13

ESA Cloud CCI Algorithm Theoretical Basis Document: Community Cloud retrieval for Climate (CC4Cl), v.6.2, 18.10.2019.

https://climate.esa.int/media/documents/Cloud_Algorithm-Theoretical-Baseline-Document-ATBD-CC4CL_v6.2.pdf

Last accessed on 07/08/2024

D14

ESA Cloud CCI Validation Report for MODIS multi-layer clouds, v1.1, 30.04.2018.

https://climate.esa.int/media/documents/Cloud_Validation-Report-CC4CL-MLEV_v1.1.pdf

Last accessed on 07/08/2024

Acronyms

Acronym

Definition

(A)ATSR

Term used to refer to the combined ATSR and AATSR dataset

AATSR

Advanced Along-Track Scanning Radiometer

ATBD

Algorithm Theoretical Basis Document

ATSR

Along-Track Scanning Radiometer

AVHRR

Advanced Very High Resolution Radiometer

BC-RMSD

Bias corrected RMSD (equal to cRMSD)

BSRN

Baseline Surface Radiation Network

C3S

Copernicus Climate Change Service

CCI

Climate Change Initiative (ESA)

CCI+

Follow-on project of ESA’s Climate Change Initiative

CDR

Climate Data Record

CDS

Climate Data Store

CEDA

Centre for Environmental Data Analysis (United Kingdom)

CERES

The NASA Clouds and Earth Radiant Energy System sensor

CF

Climate & Forecast conventions

CKB

C3S Knowledge Base

CLARA-A3

The CM SAF Cloud, Albedo and surface Radiation dataset from AVHRR data (Edition 3)

Cloud_cci

ESA’s Climate Change Initiative on Clouds

Cloud_CCI+

Extension of Cloud_cci project

CM SAF

Satellite Application Facility on Climate Monitoring

cRMSD

Centred (or bias-corrected) RMSD

DM

Daily mean

DSD

Data Set Description

DWD

Deutscher Wetterdienst (Germany’s National Meteorological Service)

ECMWF

European Center for Medium Range Weather Forecasts

ECV

Essential Climate Variable

ERA-5

ECMWF Reanalysis version 5

ERA-Interim

A global atmospheric reanalysis produced by ECMWF

EUMETSAT

European Organization for the Exploitation of Meteorological Satellites

FOV

Field Of View

GAC

Global Area Coverage (AVHRR)

GCOS

Global Climate Observing System

GOES

Geostationary Operational Environmental Satellite (NOAA)

Himawari

Japanese geostationary satellite

ICDR

Interim Climate Data Record

IFS

Integrated Forecasting System (ECMWF)

KPI

Key Performance Indicator

LUT

Lookup Table

Metop

Meteorological Operational Satellite

MM

Monthly mean

NASA

National Aeronautics and Space Administration

NEODC

National Earth Observation Data Centre (United Kingdom)

netCDF

Network Common Data Format

NISE

Near-real-time Ice and Snow Extent

NOAA

National Oceanic and Atmospheric Administration

NSIDC

National Snow and Ice Data Center

NTC

Non-Time Critical

OLCI

Ocean Land Colour Instrument on board Sentinel-3A satellite

ORAC

Optimal Retrieval of Aerosol and Cloud

PQAR

Product Quality Assessment Report

PRD 

Product Requirements Document (CM SAF)

PUM

Product User Manual (CM SAF)

RMSD

Root-mean-squared deviation

RTTOV

Radiative Transfer for TOVS

SAL

Surface Albedo

SDL

Surface Downwelling Longwave Radiation

SEVIRI

Spinning Enhanced Visible and Infrared Imager (EUMETSAT)

SIS

Surface Incoming (Downwelling) Shortwave Radiation

SLSTR

Sea and Land Surface Temperature Radiometer

SMHI

Swedish Meteorological and Hydrological Institute

SNL

Surface Net Longwave radiation

SNS

Surface Net Shortwave radiation

SOHO

Solar and Heliospheric Observatory

SOL

Surface Outgoing Longwave Radiation

SORCE

Solar Radiation and Climate Experiment

SRB

Surface Radiation Budget

SRS

Surface Reflected Shortwave radiation

Suomi-NPP

Suomi National Polar-orbiting Partnership

TCDR

Thematic Climate Data Record

TOA

Top-Of-Atmosphere

TOVS

Tiros Operational Vertical Sounder

TRGAD

Target Requirements and Gap Analysis Document

WMO

World Meteorological Organisation

List of tables

List of figures


General definitions

Climate data records

Climate data compilations from observations are most often referred to as Climate Data Records (CDRs). However, the data records from satellites may consist of different types of quantities, from original radiances to derived products. Radiance data of climate quality are defined as Fundamental Climate Data records (FCDRs) while data records consisting of satellite-derived geophysical products are defined as Thematic Climate Data Records (TCDRs). In the ideal case the TCDRs should be derived by methods using FCDRs as input. However, if standards for the used radiances have not fulfilled the strict requirements for being classified as FCDRs, these radiances may be denoted Fundamental Data Records (FDRs). Note that TCDRs can currently be based on either FCDRs or FDRs.

A special case of TCDRs are data records produced with short latency (e.g., shortly after the end of a month). These are called Interim Climate Data Records (ICDRs). The word Interim means that the data record has a higher uncertainty than the original TCDR since it has not been possible to use exactly the same input data as for the TCDR due to the short latency. Interim also means that a user may have to wait for the next edition of the TCDR to get a fully consistent and homogenous climate data record that includes data from the period with ICDR data. Normally ICDRs behave very similar to TCDRs but continuous monitoring of their quality is recommended. 

Note that since ICDRs are continuous extensions of the TCDR they are also delivered at subsequent times in separate batches (numbered 1, 2, 3... etc) where each one covers a certain time period (e.g. a number of months). Thus, when formally describing the full ICDR in the text (i.e., using the name specified in the delivery list), the ICDR version number is given but the batch number is written in generic form using letter x, for example ICDR v1.x. This is just to indicate that the batch number is only describing a temporal increment of the product and not any change of the product. 

Uncertainty parameters

The meaning of the terms uncertainty, accuracy and error is often difficult to interpret and may be treated differently in various referred documents. In this document we adopt the following interpretation:

The accuracy, uncertainty or error of an estimated ECV (or, more formally, Thematic Climate Data Record, TCDR) is described by three differently contributing components:

  1. The systematic error
  2. The random error
  3. The time-dependent error

The systematic error is commonly the mean error or the Bias. For non-Gaussian distributions of the error the median or the mean absolute error can be a more useful quantity.

The random error is commonly the root-mean-squared deviation RMSD. Sometimes the Bias is subtracted yielding the centred root-mean-squared deviation cRMSD. Notice that if the Bias is zero the two mentioned quantities are equal and may be interpreted as the standard deviation of the error (often denoted standard error).

The time-dependent error is commonly the change in Bias over time (for ECVs or TCDRs over decades). We call this parameter stability.

All TCDRs are normally evaluated against target requirements for the systematic, random and time-dependent error.

Testing the quality and consistency of TCDRs and ICDRs

This C3S project also deals with extensions of TCDRs, i.e. products derived from continued processing of the CDRs using the same methods and algorithms as originally used for TCDR production. We denote these CDRs Intermediate Climate Data Records, ICDRs. To evaluate the ICDR compliance with original TCDRs, a different approach in terms of defined requirements is followed. The ICDR is assessed on the basis of the TCDR distribution with respect to a reference validation source. After calculation of this distribution of differences, the ICDR is evaluated against the same reference and a binomial test is applied to verify that 95 % of the difference values lie within the upper and lower bounds of the TCDR difference distribution. The lower and upper bounds of the difference distribution is defined as the 2.5th and 97.5th percentiles of the difference distribution.

For further clarity, a binomial test is a way to test the statistical significance of deviations by referring to a theoretically expected distribution of observations. In this case, we use the theoretically expected distribution of observation differences which is estimated from the difference between TCDR results and corresponding results from a validation source. We now want to test if a corresponding but restricted, i.e., based on a shorter time series of ICDR results, difference distribution is similar in its shape to the original TCDR difference distribution. This can be tested by selecting one upper and one lower percentile in the original distribution (here, the 2.5th and 97.5th percentiles) and check how many samples will fall within or outside this restricted distribution if randomly extracting a number of samples. The resulting distribution of yes and no answers as a function of the number of samples can be described by the binomial distribution (see statistical standard literature for its definition). Consequently, this sample-based difference distribution from the ICDR can then be numerically compared with what could be expected from the reference distribution based on the TCDR. Based on this, one can judge whether the ICDR results are representative or not for the TCDR results. Deviations here would then indicate particular problems for the ICDR products (assuming that the character of reference observations does not change). 

More details on the estimation of errors and uncertainty parameters are given in the Report on Updated KPIs (D9).

Product requirements

Depending on the data record producer, different product requirements may be applied and they are used to evaluate validation results. An often-used way to handle this is to define several levels of requirements where each level is linked to specific needs or priorities. A three-level approach like the following is rather common:

RequirementDescription
Threshold requirement

A product should at least fulfil this level to be considered useful at all. Sometimes the term ‘Breakthrough” is used instead.

Target requirement

This is the main quality goal for a product. It should reach this level based on the current knowledge on what is reasonable to achieve.

Optimal requirement

This is a level where a product is considered to perform much better than expected given the current knowledge.


Satellite product levels

Satellite-based products are often described as belonging to the following condensed description of processing levels, each one with different complexity and information content:

LevelDescription
Level-0

Raw data coming directly from satellite sensors, often described as sensor counts.

Level-1

Data being enhanced with information on calibration and geolocation. 
Three sub-levels are often referred to:

Level-1a: Data with attached calibration and geolocation information

Level-1b: Data with applied calibration and attached geolocation information

Level-1c: Data with applied calibration and additional layers of geolocation, satellite viewing and solar angle information

Level-2

Derived geophysical variables at the same resolution and location as L1 source data.

An often-used Level-2 variety is the following:

Level-2b: Globally resampled images, two per day per satellite, describing both  ascending (passing equator from south) and descending (passing equator from north) nodes. Resampling is based on the principle that the value for the pixel with the lowest satellite zenith angle is chosen in case two or several swaths are overlapping.

Level-3

Gridded data with results accumulated over time (e.g., monthly means).

A more comprehensive definition of all processing levels is given here: https://www.earthdata.nasa.gov/engage/open-data-services-and-software/data-information-policy/data-levels.

Radiation terms

Since satellite measurements are primarily about radiation measurements in different parts of the spectrum, some definitions or synonyms need to be explained. Roughly, the spectrum is usually sub-divided into one part where solar radiation dominates and one part where radiation emitted by the Earth and the atmosphere dominates.

The solar part is usually referred to as “visible (VIS)” radiation and covers approximately wavelengths smaller than 1 µm. Two sub-regions are often referred to, namely “ultraviolet (UV)” for radiation below approximately 0.38 µm, and “near-infrared (NIR)” for radiation between 0.78 µm and 1 µm (but sometimes claimed to continue up to 2.8 µm).

The part dominated by emitted radiation from the Earth is often referred to as “thermal” radiation. Common synonyms used are “infrared (IR)” or “terrestrial” radiation. Also here, we have several sub-regions defined. The “short-wave infrared (SWIR)” region is approximately defined by wavelengths between 1 µm and 2.5 µm. The “medium-wave infrared (MWIR)” region is approximately defined by wavelengths between 2.5 µm and 5 µm. The “long-wave” region (often simply referred to as just “infrared” to represent the bulk majority of radiation emitted by the Earth) defines radiation from approximately 5 µm up to about 1 mm. Radiation above 1 mm up to 10 cm is denoted “microwave (MW)” radiation.

Special terms

The term “AVHRR-heritage” is frequently used in the TRGAD documents. By this is meant spectral channels of other sensors than the AVHRR which show a close similarity (or heritage) to the AVHRR channels, i.e., having almost the same spectral characteristics.

A product is said to be “brokered” when an existing data record from an external source (i.e., not produced exclusively within this C3S project) is handled. This also means that target requirements for these products are set to their achieved validation results since the product was not developed and validated in the C3S project.

We can get a better idea of how accurate the final product values are by using the method of “error propagation”. It means that the retrieval method is capable of accounting for errors or uncertainties in the measurements or products used to derive the final product, e.g., radiances, input or ancillary data. In this way, the uncertainty of the final products can be estimated.

Radiation fluxes are sometimes described as being “balanced”. It comes from the fact that instrument uncertainties for radiation budget measurements are often too high to be capable of providing accurate estimations of the net radiation fluxes at the top of atmosphere. Thus, balancing is a form of bias correction based on investigations of energy balance from other observations and model studies.

Calibration of radiances are sometimes described as based on “vicarious” methods. This indicates that there is no on-board mechanism on the satellite that provides the necessary calibration information. Consequently, parameters used in calibration equations have to be estimated retrospectively from historic data by use of additional references (e.g., for visible radiances, often Earth surfaces which are considered to be invariant or stable are used as reference targets).

An “OPeNDAP” server is an advanced software solution for remote data retrieval (see https://www.opendap.org/).

Triple Collocation (TC)” is a large-scale validation technique by which error variances and data-truth correlation coefficients of three independent datasets can be estimated without a specific reference observation. For further details, see Stoffelen (1998).

Scope of the document

This document provides relevant information on requirements and gaps for a total of two Surface Radiation Budget (SRB) products: The first AVHRR-based data record product is the CLARA-A3 (CLARA-A3: CM SAF CLoud, Albedo and surface RAdiation dataset from AVHRR data - Edition 3) and its associated ICDR extension. The second (A)ATSR-based data record is produced in the ESA CLOUD-CCI project and it is extended with SLSTR based products generated specifically for C3S.

The document is divided into three parts. Part 1 describes the products the present document refers to. Part 2 provides the target requirements for the products. Part 3 provides a past, present, and future gap analysis for the products and covers both gaps in the data availability and scientific gaps that could be addressed by further research activities (outside C3S).

Executive summary

The Surface Radiation Budget products consist of several components which altogether can be combined to give the total net radiation at the surface. If the latter shows a positive result, it means that the surface gains energy, which leads to a warming, while a negative result leads to a cooling. Further, surface radiation can be separated into contributions from different parts of the spectrum. Commonly, a separation into the visible part of the spectrum and the infrared part of the spectrum is applied. This separation defines largely the part where solar radiation dominates (visible) from the part where thermal radiation emitted by the Earth and the atmosphere dominate (infrared). In each of these parts, a net radiation can be calculated which in the end can be combined to give the total net radiation at the surface. A further breakdown of radiation for the two radiation parts of the spectrum is commonly made to isolate incoming radiation to the surface from outgoing radiation from the surface. If combining these two components we get the net radiation at the surface for each particular radiation region.

To summarize, this gives in total the following seven surface radiation products:

  1. Incoming solar radiation at the surface

  2. Outgoing solar radiation at the surface

  3. Net solar radiation at the surface

  4. Incoming thermal radiation at the surface

  5. Outgoing thermal radiation at the surface

  6. Net thermal radiation at the surface

  7. Net total radiation at the surface

Note that solar radiation is often referred to as “shortwave” or “visible” and that “thermal” is also often referred to as “longwave” or “infrared”.

This document describes the Surface Radiation Budget products together with their target requirements and future perspectives. They include two data records: 

  1. The CLARA-A3 data record: (CLARA-A3: CM SAF CLoud, Albedo and surface RAdiation dataset from AVHRR data - Edition 3) and its ICDR extension

  2. The ESA (A)ATSR-based v3.0 data record and its Sea and Land Surface Temperature Radiometer (SLSTR)based ICDR extension.

The CLARA-A3 TCDR comprises 42 years (January 1979-December 2020) of satellite-based climate data records, and has five radiation components. The remaining SOL and SRS fluxes can be derived as the difference between the net and longwave and shortwave downwelling components respectively. The sign convention in the CF-Standard names is that downwelling or upwelling indicates a direction of positive vector component. (downwelling positive downward, and upwelling – positive upward). The target requirements (expressed as mean absolute error) for the AVHRR-based monthly mean products are set to 5 Wm-2. However, the incoming solar radiation product is also provided as daily means and has a target accuracy of 15 W m-2. Stability requirements are set to 1 Wm-2dec-1 for both solar and thermal radiation products.

Key Performance Indicators (KPIs) for ICDR products to be produced for the extension of the TCDR (so far covering the period January 2021 until October 2024) have been defined using ERA5 monthly averaged data products as reference observations. The KPI test is based on a binomial test against low (2.5%) and high (97.5%) percentiles of the ERA5-CLARA difference distribution.

For the ESA Cloud_cci data record, the target requirements are defined by GCOS, and are defined as a mean absolute error of 1 Wm-2 for the monthly, global mean surface radiation budget in both shortwave and longwave. Stability requirements are 0.2 Wm-2dec-1 for both short- and longwave.

The data record of the (A)ATSR sensors runs from June 1995 to March 2012. The SLSTR-based ICDR data record from Sentinel-3A starts in Jan 2017 with additional Sentinel-3B data beginning in Oct 2018. Furthermore, with the start of Sentinel-3B the merged Sentinel-3A+3B product is also provided. Data has been delivered until June 2022 Options for filling the four-year gap between the end of the AATSR data record and the beginning of the SLSTR record are also provided. A key advantage of the (A)ATSR and SLSTR data (compared to AVHRR) is that the data are highly stable, both in terms of satellite orbital parameters and instrument calibration, at the cost of temporal coverage and instrument swath width.

An extensive description of past, current and future availability of data from the Advanced Very High Resolution Radiometer (AVHRR) and the (A)ATSR + SLSTR data records is given in the Gap Analysis part of this document. In addition, future prospects of utilizing AVHRR-heritage spectral channel data from new imaging sensors on new satellites are described. 

It is concluded that the AVHRR-based observations series, based on one morning and one afternoon orbit constellation, can be prolonged to reach at least 60-year duration if adding AVHRR-heritage information. However, for this to become realized, efforts are needed to harmonize and homogenize observations between true AVHRR data and AVHRR-heritage data. This concerns both calibration aspects and spatial resolution aspects. Further developments of surface radiation products are required in particular over bright surfaces, i.e., deserts and polar snow- and ice-covered areas. Work is also needed for a better characterization of conditions with high solar zenith angles and for improved estimates of fluxes under cloudy conditions. A future continuation of active observations from space is judged as crucial for further development of retrieval methods based on AVHRR-heritage data.

 Regarding the future availability of SLSTR data, the goal of the Copernicus Sentinel satellite program (jointly funded by ESA and EU) is to provide high-quality and sustained measurements for climate and environmental monitoring. Consequently, a measurement prolongation beyond the current Sentinel-3A and Sentinal-3B satellites is planned for with two more satellites (Sentinel 3C and Sentinel 3D).

1. Product description

A detailed description of the surface radiation thematic climate data records (TCDRs) follows below. They basically provide the following general set of surface radiation products which together give a complete picture of the surface radiation budget:

  1. Incoming solar radiation at the surface

  2. Outgoing solar radiation at the surface

  3. Net solar radiation at the surface

  4. Incoming thermal radiation at the surface

  5. Outgoing thermal radiation at the surface

  6. Net thermal radiation at the surface

  7. Net total radiation at the surface

Note that each TCDR might use different notations or acronyms for each individual radiation product in this list.

The following TCDRs make use of data from three different sensors: 

  1. The Advanced Very High Resolution Radiometer (AVHRR)
  2. The Advanced Along Track Scanning Radiometer (AATSR)
  3. The Sea and Land Surface Temperature Radiometer (SLSTR)

1.1 Surface Radiation Budget TCDR AVHRR CLARA v3.0 + ICDR v3.x

The CLARA-A3 TCDR comprises 42 years (January 1979 – December 2020) of satellite-based climate data records derived from the measurements of the Advanced Very High Resolution Radiometer (AVHRR) onboard the polar orbiting NOAA and METOP satellites. The subsequent ICDR production covers data from January 2021 to October 2023. . provides an overview of successive AVHRR sensors onboard dedicated platforms that are included in the generation of the CLARA-A3 dataset. The brokered CLARA-A3 TCDR and the subsequent ICDR are provided on a regular global latitude-longitude grid with a spatial resolution of 0.25° x 0.25°.

Table 1‑1: Different AVHRR versions and the lifetime of AVHRR on different platforms. The middle row for AVHRR/3 describes the satellites operated by NOAA and the bottom row the satellites operated by EUMETSAT.

Sensor

Number of channels

Platform/Lifetime

AVHRR/1

4

TIROS-N, NOAA-61979-1981

AVHRR/2

5


NOAA-7, 8, 9, 10, 11, 12, 141981-2002

AVHRR/3

6


NOAA-15, 16, 17, 18, 19

1998-onward

AVHRR/3

6


Metop-A, B, C2007-onward

The CLARA-A3 CDR includes the downward and net components of the shortwave and longwave radiation and longwave and the total surface radiation budget. The retrieval of the shortwave surface radiation parameters is based on an estimation of the atmospheric transmission and the associated reflected irradiance at the surface. The retrieval of the longwave surface radiation parameters and net fluxes is based on the data from the ERA-5 reanalysis (Hersbach et al., 2020) and cloud correction from CLARA-A3. The upward longwave radiation, necessary to derive the net thermal radiation at the surface is taken directly from ERA5.

All products are provided as monthly means, and SIS as both monthly and daily means.  shows the association between product names and Climate and Forecast Convention (CF) Standard names used in the Climate Data Store (CDS). The CF convention is an effort to create an international standard regarding the naming of geophysical products (see https://cfconventions.org/). Product names are included in the netCDF files as “long_names”, and CF names as “standard_names”. The table also indicates the links to the general surface radiation terms given in the beginning of Section 1.

Table 1‑2: ssociation table of original CLARA product names and CF Standard names used in the CDS (Climate Data Store). In the first column, product acronyms are given in parenthesis together with corresponding positions in the list of general radiation components provided in the beginning of Section 1.

CLARA Long Names

CF Standard names

Surface Downwelling Shortwave Radiation (SIS, 1)

Surface downwelling shortwave flux

Surface Downwelling Longwave Radiation (SDL, 4)

Surface downwelling longwave flux

Surface Net Shortwave Radiation (SNS, 3)

Surface net downward shortwave flux

Surface Net Longwave Radiation (SNL, 6)

Surface net downward longwave flux

Surface Radiation Budget (SRB, 7)

Surface net downward radiative flux

The EUMETSAT PyGAC AVHRR Fundamental Data Record (FDR, with the dataset doi number: 10.15770/EUM_SEC_CLM_0060 (EUMETSAT,2023a, 2023b)) is used for generation of the seven Surface Radiation Budget datasets, that are presented in the following sub-sections. The detailed description of the algorithm used to generate AVHRR GAC is given in CM SAF ATBD Cloud Products [D5], Section 2.3. Further information on the specific input and auxiliary data can be found in the Algorithm Theoretical Basis Document (ATBD) for this dataset [D1].

More specific descriptions of all surface radiation products for the CLARA family follows in the next sub-sections.

1.1.1 Surface Incoming Shortwave Radiation (SIS)

This product consists of incoming solar radiation at the surface for cloud-free and cloudy conditions. The retrieval of the surface incoming solar radiation is based on the method presented in Mueller et al., (2009). For each satellite pixel even if determined to be cloudy, the corresponding clear-sky irradiance is estimated. The satellite pixels are remapped into a regular lat-lon grid with a spatial resolution of 0.05o using nearest-neighbour remapping. The daily and monthly mean incoming surface radiation is calculated by averaging the corresponding 25 values on the final 0.25ox0.25o grid.

The full set of input and auxiliary data used to generate SIS are given in CM SAF ATBD [D1], Section 2.1.x. A flow-chart that summarizes all processing steps is given in CM SAF ATBD [D1], Figures 2-1 and 2-2.

The surface irradiance is not retrieved at larger solar zenith angles (i.e., above 80o, low-sun situations). Under the conditions of snow-covered surface and in regions with varying surface albedo, the surface incoming shortwave radiation dataset performs poorly in comparison to the reference datasets. Grid points in such areas are masked out, and so, large parts of the polar regions are not covered by CLARA-A3. Moreover, the determination of the exact number of the required pixels in each grid box (20) to derive a meaningful daily average and the number of valid daily means (20) to derive a meaningful monthly average needs further investigation. In the period 1979–1991, when only afternoon satellites with AVHRR instruments were operational, some parts of the globe did not have the required number of observations per months, resulting in gaps in the spatial coverage. Moreover, in the case of instrument errors or satellite transitions, the period from 1979 to 2003 is influenced by measurement gaps of one or more AVHRR instruments, resulting in a reduced accuracy estimation of daily and monthly radiation fluxes.

A full list of the known limitations and their implications for SIS are described in CM SAF ATBD [D1] Chapter 2.1.2.4. In summary, these are:

  • Limitation due to the temporal resolution of surface albedo 
  • Uncertainty under cloudy conditions, especially with thin clouds
  • Uncertainties in the cloud-detection algorithm
  • Application of monthly climatological aerosol information instead of the real time data

1.1.Surface Downwelling Longwave Radiation (SDL)

The thermal or longwave fluxes are decoupled from the visible and infrared radiation at the top-of-atmosphere. Thus, satellite data alone do not contain sufficient information to retrieve the longwave radiation at the surface without additional data sources. The ERA-5 reanalysis data together with the cloud correction factor from CLARA-A3 is therefore used.

The surface longwave radiation in the CLARA-A3 is only provided as monthly averages. The algorithm used to generate the SDL dataset is described in CM SAF ATBD [D1] Section 2.2.1. 

The known limitations and their implications for SDL are described in CM SAF ATBD [D1] Section 2.2.1.3. These are:

  • Underestimation of inter-annual cloud cover variability due to linear regression

  • Limited topographic correction by using constant values (i.e., -2.8 Wm-2 per 100 m elevation).

1.1.3 Surface Net Shortwave Radiation (SNS)

The SNS product calculates the net shortwave radiation received at the Earth's surface. It subtracts the amount of outgoing (reflected) solar radiation (SRS) from incoming solar radiation (SIS). Alternatively, it's expressed (Eq. 1) as the product of SIS and the fraction of incoming radiation not reflected, which is (1 - SAL).

The SNS product is generated from the SIS daily averages and the pentad-average blue-sky surface albedo (SAL) monthly means, both of which are part of the CLARA-A3 data record.



\( SNS = SIS - SRS = SIS (1- SAL) \)

(Eq.1)

The set of input and auxiliary data used to generate the SIS is given in CM SAF ATBD Surface Radiation Products [D1], Section 2.1.1. The set of the input and auxiliary data used to generate the SAL is given in CM SAF ATBD Surface Albedo [D10], Section 3.1.3.

1.1.4 Surface Net Longwave Radiation (SNL)

The SNL product is generated from the Surface Downwelling Longwave radiation (SDL) brokered from EUMETSAT’s CM SAF CLARA-A3 and the Surface Outgoing Longwave radiation (SOL) derived from the monthly net surface longwave and the surface downwelling longwave radiation provided, both provided as part of ERA-5.

The set of input and auxiliary data used to generate the SDL is given in CM SAF ATBD Surface Radiation Products [D1], Section 2.2.1.

1.1.5 Surface Radiation Budget (SRB)

The SRB product is composed as the sum of SNL and SNS.

1.2 Surface Radiation Budget TCDR CCI_AATSR v3.0 + ICDR CCI_SLSTR v3.1.1-v4.0

This section describes the surface radiation products of the TCDR CCI_AATSR v3.0 data record, which we will refer to as Cloud_cci v3, and its extension with the same retrieval scheme applied to the Sea and Land Surface Radiometer (SLSTR), which is produced specifically for C3S.

The Cloud_cci v3 data record is based on Along-Track Scanning Radiometer 2 (ATSR-2) and Advanced Along-Track Scanning Radiometer (AATSR) observations onboard the ESA 2nd European Research Satellite (ERS-2) and ENVISAT satellites. Together, the data record provided by these two instruments is often abbreviated to (A)ATSR. The SLSTR instrument, which is the successor to (A)ATSR, is on board the Copernicus Sentinel-3 platform [D11].

The SLSTR instrument was designed as the operational successor to the (A)ATSR instruments, using the same measurement principles and techniques, improving them based on the experience gained with the (A)ATSRs, and continuing the 17-year data record provided by ATSR-2 and AATSR (21 years if ATSR-1 is included). Unfortunately, the development time of the Copernicus Sentinel satellites and the demise of ENVISAT in 2012, broke the continuity of this dataset, with an almost five-year gap between the end of the AATSR record and the availability of SLSTR. Despite this, SLSTR products can be considered an ICDR extension of the (A)ATSR TCDR, for the following reasons:

  1. The (A)ATSR and SLSTR instruments were conceived with the goal of creating long-term data records for climate monitoring. Consistency and stability are at the core of their design.

  2. The instruments are very similar – SLSTR provides a wider swath, some additional channels, increases the spatial resolution of the shortwave channels and alters the viewing geometry compared to (A)ATSR. But the differences in the instrument and orbital characteristics between AATSR and SLSTR are comparable to those between ATSR-2 and AATSR.

  3. There are two SLSTR instruments, one onboard Sentinel-3A and -3B respectively. If data from both platforms are included in producing level-3 products, SLSTR provides almost global coverage twice daily (from both the daytime and night time overpasses). The v4.0 SLSTR data is just such a product, while v3.1.1 provides data from each instrument separately.

Observations are available on a 1x1 km grid, which closely matches the true instrument spatial resolution globally and the final CDR is compiled in a regular global grid with 0.5 latitude-longitude resolution for monthly averages. The covered time period of the Cloud_cci data record ranges from June 1995 to April 2012. SLSTR products from Sentinel-3A begin in January 2017, with Sentinel-3B products beginning in October 2018. At the time of writing SLSTR data from both platforms is available up to June 2022, with data for 2022 to be published shortly.

 Cloud_cci products were based on the third reprocessing of the AATSR-multimission archive (denoted collection 3), which included vicarious calibration of the shortwave channels over the entire data record to correct for long-term calibration drift (Smith, 2012). SLSTR products are based on collection 3 of the non-time critical (NTC) SLSTR level 1 archive. NTC products are products delivered with a latency of 1 month or longer.

The Cloud_cci v3 data record was produced using the Community Cloud for Climate (CC4Cl) processing chain, which is based on the Optimal Retrieval of Aerosol and Cloud (ORAC) retrieval scheme, both of which are described in detail in [D12, D13] and by Sus et al. (2018) and McGarragh et al. (2018).

In addition to being based on (A)ATSR radiances, the Cloud_cci CC4Cl processing chain makes use of the following auxiliary datasets: 

  • USGS Digital Elevation Map (USGS, 1996)

  • ERA-Interim surface and atmospheric profile temperatures and pressure (Dee et al., 2011)

  • ERA-Interim profiles of moisture content and Ozone concentrations (Dee et al., 2011)

  • ERA-Interim snow depth and albedo (Dee et al., 2011)

  • National Snow and Ice Data Center (NSIDC) Near-real-time Ice and Snow Extent (NISE) sea ice concentration (Brodzik and Stewart, 2016).

  • ERA-Interim 10 m u and v wind components (Dee et al., 2011).

  • MODIS-based land surface bidirectional reflectance distribution function (BRDF) data (MCD43C1 Collection 6, (Schaaf and Wang, 2015)).

  • Land surface emissivity from the Cooperative Institute for Meteorological Satellite Studies (CIMSS) “Baseline Fit” database.

  • Solar and Heliospheric Observatory (SOHO) and Solar Radiation and Climate Experiment (SORCE) incoming total solar irradiance.

 For Level-1 data outside the temporal coverage of the above auxiliary datasets (for example, ATSR-2 data prior to the 1999 launch of MODIS products), climatologies based on the above auxiliary datasets are used. In the case of the SLSTR ICDR, ERA-5 data is used rather than ERA-Interim (for details, see [D14]).

The seven surface radiation parameters provided to C3S from the Cloud_cci v3 dataset and SLSTR are the same as those provided by the AVHRR products described in section 1.1.2 and they have similar limitations.

2. User Requirements

This section describes the requirements which have been set to be achieved by the described products. Requirements can be set at different levels (as explained in the section with General definitions) but here we will focus on what is called the Target Requirements. These requirements define the main goals for data producers which have to be fulfilled by their products. Requirements are specified by the use of various accuracy parameters which are also listed in the section with General definitions. Observe that for brokered products the target requirements are set to the achieved validation results since these products are not developed and tested within the C3S project.

Concerning products to be used in climate monitoring, requirements for what should be achievable through Earth Observation systems are generally defined by the World Meteorological Organisation (WMO) Global Climate Observation System (GCOS) expert panel. However, these requirements are generally oriented towards the capability and resolution of climate models with a rather course spatial resolution while many products listed here are focusing more on the monitoring of local and regional scale conditions. Also, they are often not attainable using existing or historical observing systems. Thus, GCOS requirements are not always identical to the requirements listed here since also user groups other than the climate modelling community have contributed to setting the requirements. However, the relationship to GCOS requirements is discussed below for each individual product. Note also that the GCOS requirements are not specified for all products described in this document. 

2.1 Surface Radiation Budget TCDR AVHRR CLARA v3.0 + ICDR v3.x

2.1.1 Summary of target requirements (Key Performance Indicators – KPIs)

This report covers the following products from CLARA family: the SIS, the SDL, the SNS, the SNL and the SRB. The predefined requirements for these products are given in CM SAF Product Requirement Document (PRD) [D4], Annex A. The validation of the three core surface radiation datasets was conducted against surface measurements from the Baseline Surface Radiation Network (BSRN) (Ohmura et al., 1998).

There are three accuracy categories in the CM SAF PRD document ([D4], Section 5): threshold, target and optimal accuracies (see General definitions for a detailed description). They are defined keeping in mind different target users: operational climate monitoring, global and regional climate modelling and global and regional climate studies. 

The CLARA-A3 surface radiation products are brokered from the CM SAF project, and consequently cannot be altered in this C3S project Therefore, the current target requirements on the key performance of the data set, measured within C3S by the so-called Key Performance Indicators (KPIs), are defined as the results achieved (with original requirements described in D3 and D4) in previous CLARA-A3 validation activities in CM SAF. These values are listed in Table 2-1.

Table 2‑1: Key Performance Indicators (KPIs) or target requirements for surface radiation products.

Variable

KPI: accuracy Wm-2
(mean absolute error)
Fulfilled by CLARA-A2.1 CDR

KPI: decadal stability,
Wm
-2decade-1
Fulfilled by CLARA-A2.1 CDR

SIS Monthly mean (MM)

7.3 Wm-2

1 Wm-2decade-1

SIS Daily mean (DM)

16.9 Wm-2

1 Wm-2decade-1

SDL MM

7.2 Wm-2

1 Wm-2decade-1

SNS MM

10.8 Wm-2

1 Wm-2decade-1

SNL MM

6.8 Wm-2

1 Wm-2decade-1

SRB MM

9.7 Wm-2

1 Wm-2decade-1

At the time (~2018) when requirements for the CLARA-A3 data record were defined, as described in [D4], there was no guidance available for surface radiation products in the available GCOS-107 document [D6]. Instead, requirements had to be set based on a dialogue with experts and potential users (e.g., in association with CM SAF User Workshops).

For the evaluation of the ICDR, corresponding products from the ERA-5 reanalysis are used as reference. The distribution of the global differences between ERA5 products and the CLARA-A3 TCDR has been compiled and the corresponding 2.5 and 97.5 percentile differences are given in Table 2-2. These percentiles are used to check, by means of a binomial test at 5 % significance level, whether the corresponding ICDR differences are consistent with the TCDR differences or not. Further details on these tests are found in the Report on Updated KPIs (D9).

Table 2‑2: Key Performance Indicators (KPIs) for the surface radiation budget ICDR products.

Variable

KPI: lower percentile

(2.5 %), Wm-2

KPI: higher percentile

(97.5 %), Wm-2

SIS monthly

-7.95 W/m²

16.08 W/m²

SDL monthly

-0.44 W/m²

0.38 W/m²

SNS monthly

-6.77 W/m²

14.18 W/m²

SNL monthly

-0.51 W/m²

0.38 W/m²

SRB monthly

-5.82 W/m²

10.49 W/m²

2.1.2 Discussion of requirements with respect to GCOS and other requirements

The product requirements listed in the CM SAF Product Requirement Document [D4] for the CLARA-A3 data record were generally defined in accordance with the GCOS report GCOS-107 [D6]. However, since this version of the GCOS document did not yet include requirements for the Surface Radiation ECV, some CM SAF-specific requirements had to be defined and used as explained above in section 2.1.1. A comparison between original GCOS requirements and achieved CLARA-A3 results is given in Table 2-3.

Table 2‑3: GCOS-200 requirements for surface radiation products compared to CLARA-A2.1 requirement.

Requirements

GCOS (Target) 

CLARA-A2.1 TCDR + ICDR v2.x 

Spatial resolution

100 km

25 km

Temporal resolution

Monthly (resolving diurnal cycle)

Monthly

Accuracy:

SIS MM
SIS DM
SDL MM
SNS MM
SNL MM
SRB MM


1 Wm-2
1 Wm-2
1 Wm-2
1 Wm-2
1 Wm-2
1 Wm-2


7.3 Wm-2
16.9 Wm-2
7.2 Wm-2
10.8 Wm-2
6.8Wm-2
9.7 Wm-2

The GCOS requirements for the ECV Surface Radiation Budget are summarized in GCOS-200 [D7] and include requirements for the horizontal resolution, temporal resolution, accuracy and stability. However, these requirements are only valid for the net fluxes (i.e., SNS and SNL) and not for all individual radiation budget components (see Table 2-4 below). However, one could claim that individual radiation budget components should consequently be constrained in the same way as net fluxes.

All products in the brokered CLARA-A3 dataset fulfil the new GCOS requirements regarding the horizontal and temporal resolution. However, the SNS and SNL products do not fulfil the new requirements on accuracy and stability which are very stringent compared with previously used requirements for the CLARA data record. Nevertheless, achieved accuracy and stability results allow consistent quantification of mean values, anomalies, variability and the Earth energy budget in general. Existing uncertainties in the methodology of comparison with area-to-point measurements (i.e. satellite-area to point-ground-based reference networks) are important reasons for not fulfilling the new requirements (as discussed by Schwarz et al., 2017).

2.1.3 Data format and content issues

Information on the file format is provided in the CM SAF Product User Manual (PUM) [D2] Section 4. The CLARA-A3 surface radiation products are defined using standard data formats (NetCDF4) and map projections (regular latitude/longitude). Meta data and naming definitions follow the Climate & Forecast (CF) conventions (https://cfconventions.org).

Based on the recommendations formulated within C3S_312b Lot1, the license field was added to all the netCDF-files that are brokered and produced. The goal is to provide a clear identification of the data record producer.

2.2 The Surface Radiation Budget TCDR CCI_AATSR v3.0 + ICDR CCI_SLSTR v3.1.1+v4.0

2.2.1 Summary of target requirements

The GCOS expert panel defines and lays down targets for the observation of ECVs (Table 2-4).

The Cloud_cci product, and the SLSTR extension, achieve or exceed the frequency and resolution requirements, with the exception of resolving the diurnal cycle, which is not possible with a single low-Earth-orbit platform. The GCOS accuracy target of 1 Wm-2 is not met by the Cloud_cci/SLSTR products, however it should be noted that GCOS requirements are targets for what should be achievable through Earth observation and are often not attainable using existing or historical observing systems. GCOS only defines targets for the net-radiation fluxes, but as these are simple sums of the up- and down-welling fluxes, they provide target constraints for the entire suite of surface radiation parameters provided by the Cloud_cci products.

Table 2‑4: Target requirements for surface radiation budget defined by GCOS-200 ([D7], Table 23, page 279].

GCOS quantity

Corresponding Cloud_cci variable

GCOS targets

Surface ERB longwave

Surface net longwave radiation

(SNL)

  • Frequency: Monthly (resolving diurnal cycle)

  • Resolution: 100 km

  • Measurement uncertainty: 1 Wm-2 on global mean

  • Stability: 0.2 Wm-2dec-1

Surface ERB shortwave

Surface net solar radiation

(SNS)

  • Frequency: Monthly (resolving diurnal cycle)
  • Resolution: 100 km

  • Measurement uncertainty: 1 Wm-2 on global mean

  • Stability: 0.2 Wm-2dec-1

2.2.2 Key Performance Indicators - KPIs

Similar to the CLARA-A3 products, the Cloud_cci (A)ATSR products are brokered products (but in this case from the ESA CCI programme) and cannot be altered within the scope of C3S_312b_Lot1 and C3S2_312a_Lot1. Table 2-5 shows the target requirements based on earlier validation results. The performance of the ESA-CLOUD-CCI product was assessed against ground-based radiation measurements of downwelling flux from the Baseline Surface Radiation Network (BSRN) for surface incoming shortwave radiation (SIS) and surface downwelling longwave radiation (SDL), while the other parameters were compared to values derived from the CERES SRB CDR. Stability figures were only calculated for SIS and SDL comparisons, where ground-truth data is available.

Table 2‑5: Key performance indicators (KPIs) or target requirements (i.e. fulfilled requirements by the ESA-CLOUD-CCI project) for surface radiation products from ESA_CCI_AATSR TCDR v3.0.

Variable

KPI: accuracy (Bias)

Fulfilled by ESA-CLOUD-CCI

KPI: decadal stability

Fulfilled by ESA-CLOUD-CCI

SIS

8.2 W/m2

0.97 W/m2/decade

SDL

12 W/m2

2.76 W/m2/decade

SRS

4.6 W/m2

-

SNS

13 W/m2

-

SOL

11 W/m2

-

SNL

23 W/m2

-

SRB

36 W/m2

-

The Cloud_cci (A)ATSR data record forms the basis for the calculation of KPIs for the SLSTR based ICDR. The KPIs for the SRB products are based on comparison against the NASA Clouds and the Earth's Radiant Energy System (CERES) instruments (Wielicki et al., 1996). These comparisons are represented as the 2.5 and 97.5 percentiles of the distribution of differences between (A)ATSR or SLSTR monthly-mean values and the corresponding CERES values (corrected for the mean seasonal cycle). These values, calculated from the 12-year (A)ATSR CDR are summarized in Table 2-6.

It should be noted that CERES surface radiation values are a product of a similar level of processing, based on knowledge or assumptions of the atmospheric state, as those produced from the CC4Cl retrieval scheme. Thus, CERES should not be considered as a more accurate estimate of SRB than the Cloud_cci CDR. However, significant effort has gone into ensuring the stability and consistency of the CERES products, making them suitable for monitoring the relative performance of the Cloud_cci products and their extension with SLSTR.

Table 2‑6: Key performance indicators (KPIs) for the Cloud_cci SRB record, applied to the SLSTR ICDR data.

Variable

KPI: lower percentile

(2.5 %), Wm-2

KPI: higher percentile

(97.5 %), Wm-2

SIS Monthly mean

-1.3

2.12

SRS Monthly mean

-0.45

0.36

SDL Monthly mean

1.95

2.23

SOL Monthly mean

-4.04

3.68

2.2.3 Discussion of requirements with respect to GCOS and other requirements

A discussion on these requirements has already been provided in section 2.2.1 (related to Table 2-4).

2.2.4 Data format and content issues

The Cloud_cci v3 cloud property products are defined using standard data formats (netCDF) and map projections (regular latitude/longitude grids). Meta data definitions follow the Climate & Forecast conventions (http://cfconventions.org/).

3. Gap Analysis

3.1 Description of past, current and future satellite coverage

3.1.1 Surface Radiation Budget TCDR AVHRR CLARA v3.0 + ICDR v3.x

The surface radiation and cloud products belong to the CLARA-A3 datasets. These two datasets make use of the same instruments, installed on the same satellites. A full presentation of the AVHRR sensor and relevant satellites can be found in the C3S2_D3.1.1-2023-CLD, Section 3.1.1 Cloud properties TCDR AVHRR CLARA v3.0 +ICDR v3.x [D8].

Table 3-1: Known data gaps.

Product
name

CLARA-A3

Monthly

SIS daily

SIS

1985-02

1979-02-22 - 1979-02-24
1979-10-01 - 1979-10-07
1979-11-03 - 1979-11-04
1979-11-07
1979-11-17 - 1979-11-18

1980-02-22 - 1980-02-27
1980-03-15 - 1980-03-20
1980-03-30 - 1980-04-02
1980-06-26 - 1980-06-29
1980-07-12 - 1980-07-19
1980-12-12 - 1980-12-18

1981-05-09 - 1981-05-11
1981-08-01 - 1981-08-03
1981-08-22

1982-05-29 - 1982-05-31
1982-09-25 - 1982-09-26

1983-07-27 - 1983-08-02
1983-08-06

1984-01-14 - 1984-01-15
1984-12-06

1985-02-02 - 1985-02-24
1986-03-15

SDL

1981-08
1985-02

SNS

1980-03
1985-02

SNL

1981-08
1985-02

SRB

1980-03
1981-08
1985-02

3.1.2 (A)ATSR-based Surface Radiation Products and its SLSTR-based ICDR extension

The Cloud_cci v3 is based on radiances provided by the ATSR series of sensors. These instruments flew on sun-synchronous polar orbiting satellites with daytime equatorial crossing times in the mid-morning; 10:30 Local Time on Descending Node (LTDN) for ATSR-2 and 10:00 LTDN for ENVISAT, with both satellites sharing the same ground track. There were 14.3 orbits per day, meaning 28 equatorial overpasses per-day, with measurements covering a total of 18% of equatorial circumference of the Earth (with equally spaced 512 km swaths). The observation frequency increased at higher latitudes (with a maximum of 14 observations per day at the poles) due to increasing overlaps between the satellite swaths. Both sensors provided the same seven channels (and used the same conical dual-viewing geometry), but not all channels were provided at all times, or at full digitization rate, from ATSR-2, due to limitations of the data bandwidth provided by the ERS-2 platform. Over ocean regions, ATRS visible channels were often only provided in a 256 pixel “narrow-swath” mode. The channels provided by both instruments were centred at 0.55, 0.67, 0.87, 1.6, 3.7, 10.8, 12.0 m and the filter band passes were very similar between instruments. Despite the low-data rate modes of ATSR-2, the combination of the very similar instrument specifications, very close orbital parameters and the lack of any significant orbital drift in the ERS-2 and ENVISAT satellites mean that ATSR-2 and AATSR provide a highly consistent data record, especially when compared to that provided by the AVHRR record used by the CLARA-A2 TCDR (although AVHRR provides a much longer data record).

The TCDR from Cloud_cci v3 begins with the launch of ERS-2 in mid-1995 and continued until the failure of ENVISAT in April 2012. Due to instrument problems, there is a six-month data gap in the ATSR-2 record from January to June 1996.

There is an overlap of 1 year of data between the two platforms, between mid-2002 (when ENVISAT was launched) and mid-2003 (when the onboard data storage on ERS-2 failed). There is additional ATSR-2 data available up-to 2009, but this is not global as data could only be collected when the satellite was within line-of-sight with a ground receiving station, and has not been included in the TCDR. There is some scope to push the coverage of the ATSR cloud record back to 1991, by using the ATSR-1 instrument (onboard ERS-1), which also flew in a similar orbit to its successors. However, ATSR-1 lacked the shortwave channels (apart from the 1.6 m) channel, which would reduce the information available to daylight retrievals and would represent a significant inhomogeneity in the TCDR.

The extension of the ATSR TCDR (i.e., the SLSTR ICDR) makes use of SLSTR sensors onboard the Sentinel-3 platforms. SLSTR represents a significant upgrade over (A)ATSR, providing a wider swath, two satellites within interleaved orbit swaths, additional channels and the data available security of an operational system. The Sentinel-3s have a very similar orbit to ENVISAT and the ERS satellites, with a sun-synchronous orbit with an LTDN of 10:00, and 14.3 orbits per-day. However, there is slightly longer than a 4-year gap between the end of the (A)ATSR record and the first SLSTR data. There are several options available to fill this gap, as ORAC can be applied to most radiometers with similar channels to those provided by ATSR. Indeed, cloud CDRs of ORAC applied to both MODIS and AVHRR already exist, having been produced in the Cloud_cci program, but have not been brokered to the CDS.

We suggest that addressing the (A)ATSR/SLSTR data gap, by linking the two datasets with a third source of data which overlaps with each should be a priority for Copernicus, and would likely be readily achievable given the necessary support. There are several datasets which provide similar measurements to the (A)ATSR/SLSTR SRB dataset (including the other C3S products discussed in this report) and which overlap both (A)ATSR and SLSTR temporally. Such data could be used to assess the consistency of the (A)ATSR TCDRs and SLSTR ICDRs, but this remains to be done for the SRB dataset. 

It should also be noted that ORAC could be applied to the VIIRS and MetImage instruments described above, which could complement the SLSTR ICDR.

Regarding the future availability of SLSTR data, a measurement prolongation beyond the current Sentinel-3A and Sentinal-3B satellites is planned with two more satellites (Sentinel 3C and Sentinel 3D, which are scheduled for launch in 2024 and 2028, respectively).

3.2 Development of processing algorithms

The surface radiation and cloud products belong to the CLARA-A3 TCDR and use the same satellite sensors from the AVHRR-family. Thus, the pre-processing methods are the same for these two datasets, thereby ensuring the quality checks and corrections of the radiances and a homogeneity of the input data series. A full description of these issues can be found in the C3S2_D3.1.1-2023-CLD, Section 3.2.1 Cloud properties TCDR AVHRR CLARA-A3 + ICDR v3.x [D8]. Specific issues and research needs of the surface radiation datasets are described later in this document (Section 3.5).

3.2.1 Surface Radiation Budget TCDR AVHRR CLARA v3.0 + ICDR v3.x

The Surface Downwelling Longwave and products use the ERA-5 data to define the surface downwelling longwave fluxes. This dependence limits the applicability of the SDL, SNS, and the SRB data records for the evaluation of model simulations. Also, ERA5 reanalysis data cannot be considered temporally homogeneous over the full available time period due to the change of the observing system that is used in the assimilation scheme of the reanalysis. This will result in an inhomogeneity of the CLARA-A3 data record of the longwave radiation and the surface radiation budget.

3.2.2 (A)ATSR-based Surface Radiation Products and their SLSTR-based ICDR extension

As with the CLARA CDRs, the stability and quality of the input data is the key parameter which influences the reliability of the Cloud_cci v3 CDRs. The (A)ATSR TCDR is based on version 3 of the “AATSR multimission archive” maintained by Center for Environmental Data Analysis (CEDA) and the UK National Earth Observation Data Centre (NEODC). This record incorporates the latest calibration corrections (including long-term drift corrections from vicarious calibration) and represents the most consistent and accurate record of radiances from the (A)ATSR record. A future update to this record would make a reprocessing of the Cloud_cci TCDR possible.

In the case of the SLSTR ICDR, the status of the level 1 radiances is considerably less stable. Data from early in the SLSTR record has considerably worse calibration and geolocation than more recent data. When a fully reprocessed version of the data record becomes available in the future, regeneration of the cloud ICDR would be possible.

EUMETSAT has provided updated calibration corrections to SLSTR shortwave channels, communicated through the Sentinel-3 Scientific Validation Team (S3VT), which have been applied retrospectively. However, there is not yet any information on the stability of the SLSTR calibration over time and there remain issues with the colocation of SLSTR channels in early versions of the level 1 products.

3.2.2.1 Adaptions of the ORAC scheme to better exploit SLSTR

As mentioned above, SLSTR provides some additional channels over the earlier AATSR instruments. Of particular note is the new 1.3 m channel, which, due to its location in a water-vapour absorption feature, is particularly sensitive to the presence of high-altitude clouds. Utilizing this channel in the retrieval scheme itself is unlikely to be beneficial, as accurate knowledge of the water vapour profile is needed to accurately model the radiances. However, the use of this channel in prior cloud-detection and characterization has been studied under the Cloud_cci+ project and an assessment of its impact on the quality of ORAC cloud retrievals is underway.

3.2.2.2 Forward model improvements

Further improvements to the forward modelling of clouds for the ORAC retrieval scheme are also underway. In particular:

  • The SLSTR ICDR makes use of ERA-5, rather than the ERA-Interim used for the TCDR.

  • The spectral dependence of cloud scattering and absorption will be modelled across the bandpass of the instrument channels (rather than at the channel centre as previously).

  • At present cloud is modelled as an infinitesimally thin layer within an atmosphere modelled by RTTOV. The modelling of cloud geometric thickness effects will also be investigated in the upcoming Cloud_cci+ project.

  • The use of new ice cloud optical properties will also be investigated, as these become available.

  • Improvements in the propagation of uncertainty from L2 products to gridded L3 products is also under investigation.

3.3 Methods for estimating uncertainties

3.3.1 Surface Radiation Budget TCDR AVHRR CLARA v3.0 + ICDR v3.x

The current CLARA-A3 products are not associated with any uncertainty estimates. Thus, uncertainty information is only available as results achieved by associated validation activities ([D3]). The comparison between point (BSRN) and gridded data brings additional uncertainties, including those due to the representativeness of the surface measurement location. The coverage factor, measure of representativeness of point data for the surrounding area, for the uncertainty of the BSRN measurements is being developed. The coverage factor was introduced by Immler et al., 2010. Work is underway to add and improve the uncertainty characterisation for CLARA-A3 measurements based on these factors in the future CLARA releases.

3.3.2 (A)ATSR-based Surface Radiation Products and their SLSTR-based ICDR extension

The ORAC retrieval scheme provides propagated uncertainties on the retrieved cloud parameters, but these are not propagated through the broadband flux calculations at present. Thus, no uncertainty estimates are provided in the Cloud_cci SRB products, aside from the standard deviation of the level-2 pixels included in each monthly-mean grid box. Thus, as with CLARA-A2.1 products, uncertainty information is only available through validation activities.

3.4 Opportunities to improve quality and fitness-for-purpose of the CDRs

3.4.1 Surface Radiation Budget TCDR AVHRR CLARA v3.0 + ICDR v3.x

The future edition of the CLARA dataset, CLARA-A3.5 (to be released in 2026), will cover an extended period from 1979 to 2024. Adding these years to the currently brokered dataset is of great value for climate studies. CLARA-A3.5 will also provide improved cloud detection and a revised AVHRR calibration. This will lead to an improved quality of surface radiation products for the entire time period covered.

3.4.2 (A)ATSR-based Surface Radiation Products and their SLSTR-based ICDR extension

Most potential improvements to ORAC radiative flux products stem from improvements to the underlying cloud retrieval scheme, which are discussed below.

The planned development of the ORAC retrieval scheme, as applied to (A)ATSR and SLSTR, has already been described in section 3.2.2. ORAC is under active development, both through the ESA CCI+ program and through national UK funding (in particular, under the National Centre for Earth Observation). New improvements of the scheme, where applicable, will be fed through to the production of improved CDR products from SLSTR.

It is also worth noting that the ORAC scheme is not specifically designed for application to (A)ATSR or SLSTR. CDRs have already been produced using the scheme for the AVHRR and MODIS instruments, under previous iterations of the CCI program. The scheme has also been applied to geo-stationary sensors (Spinning Enhanced Visible and Infrared Imager - SEVIRI, GOES Advanced Baseline Imager – ABI and Himawari Advanced Himawari Imager - AHI), and improved application of the scheme to SEVIRI in particular (making use of the water-vapor sounding channels provided by the instrument) is being undertaken in CCI+.

 The code includes the ability to utilize sounding channels (CO2 slicing and water-vapor absorption), as well as a multi-layer cloud retrieval mode ([D16]), which greatly improve on the shortcomings of the existing “heritage channel” (AVHRR-like) CDRs produced in CCI, and retrieves the properties of dual-layer cloud scenes. Thus, the scheme provides the scope for the production of cutting-edge CDRs from a wide range of instruments, all with a consistent retrieval approach.

3.5 Scientific Research needs

3.5.1 Surface Radiation Budget TCDR AVHRR CLARA v3.0 + ICDR v3.x

3.5.1.1 Surface Incoming Shortwave Radiation (SIS)

The known issue is poor performance of SIS compared with reference datasets in regions with highly-reflecting surfaces. For this reason, studies in relation to snow detection are needed. Additional studies with respect to satellite observations under twilight/dawn conditions could improve the data quality of prior satellite generations.

It should be noted that these and other issues will be tackled and improved in future revisions/versions of the CLARA dataset. 

Furthermore, retrieval of the SIS under the cloudy conditions requires the broadband shortwave flux estimate. In the current version, this conversion is done using two satellite channels. Further investigations are needed on the effect of such a conversion on the retrieved SIS values. The topic has been discussed further by Akkermans et al. (2020).

3.5.1.2 Surface Outgoing Longwave Radiation (SOL)

An issue for all satellite measurements is the estimate of the surface emissivity. It is desirable to have routinely updated emissivity estimates. Such data is available and continuously updated from MODIS products and they are likely to be achieved also from new sensors in the future. Thus, this problem can be addressed for more recent datasets but the problem remains for products covering the period before MODIS (i.e., before 2000). 

3.5.2 (A)ATSR-based Surface Radiation Products and their SLSTR-based ICDR extension

The requirements for the further improvements of the (A)ATSR and SLSTR CDRs are identical to those for the AVHRR CLARA CDRs.

3.6 Opportunities from exploiting the Sentinels and any other relevant satellite

3.6.1 Surface Radiation Budget TCDR AVHRR CLARA v3.0 + ICDR v3.x

Surface Radiation Budget products are brokered without changes. In the current version no information from the Sentinel satellites is used. However, the Surface Radiation Budget ESA SLSTR ICDR v3.x dataset is directly benefiting from the SLSTR instrument installed on the Sentinel-3 series satellites. This instrument has almost identical channels as AVHRR and also some unique channels. This gives a possibility for its use for inter-comparison.

Future CLARA editions will use non-AVHRR satellite instruments in the CLARA data record, e.g., VIIRS, to ensure the continuation of the CLARA data record into the future.

3.6.2 (A)ATSR based Surface Radiation Products and their SLSTR-based ICDR extension

The ESA SLSTR v3.x ICDR directly exploits data from the Sentinel-3 platform. There have been examples shown of utilizing Sentinel-3 OLCI-like measurements (mainly using MERIS on ENVISAT) for cloud retrieval in conjunction with (A)ATSR or SLSTR (Carbajal Henken et al. 2014), but difficulties in cross-calibration and co-registration of the different instruments have meant these products have not shown improved performance over the (A)ATSR/SLSTR only algorithms. The availability of a well co-located and calibrated joint SLSTR-OLCI L1 product, could resurrect this approach to further improving cloud products derived from Sentinel-3 (and the preceding ENVISAT).

As discussed in section 3.1.2, the ORAC retrieval scheme can be, and has been, applied to a wide range of satellite visible-IR imaging radiometers. A particular instrument, of direct relevance to the Sentinel satellite program is the Flexible Combined Imager (FCI) to fly on Meteosat Third Generation/Sentinel-4. This instrument is essentially a replacement for the SEVIRI sensors on MSG, with capabilities similar to those provided by Himawari-AHI and GOES-ABI imagers (which ORAC has already been applied to).

References

Akkermans, Tom, and Clerbaux, N., 2020 : Narrowband-to-broadband conversions for top-of-atmosphere reflectance from the Advanced Very High Resolution Radiometer (AVHRR). Remote Sensing 12, no. 2 (2020): 305, DOI: https://doi.org/10.3390/rs12020305

Brodzik, M. J. and J. S. Stewart. 2016: Near-Real-Time SSM/I-SSMIS EASE-Grid Daily Global Ice Concentration and Snow Extent, Version 5. Boulder, Colorado USA. NASA National Snow and Ice Data Center Distributed Active Archive Center. doi: https://doi.org/10.5067/3KB2JPLFPK3R.

Carbajal Henken, C.K., R. Lindstrot, R. Preusker and J. Fischer, 2014: FAME-C: cloud property retrieval using synergistic AATSR and MERIS observations. Atmos. Meas. Tech., 7, 3873–3890, DOI:https://doi.org/10.5194/amt-7-3873-2014.

EUMETSAT, 2023a: AVHRR Fundamental Data Record – Release 1 – Multimission, European Organisation for the Exploitation of Meteorological Satellites [data set], 2023, DOI: https://doi.org/10.15770/EUM_SEC_CLM_0060

EUMETSAT, 2023b: PyGAC AVHRR FDR Release 1 – Product Users Guide, Document EUM/OPS/DOC/21/1269950, DOI: https://doi.org/10.15770/EUM_SEC_CLM_0060

Hersbach, H., Bell, B., Berrisford, P., Hirahara, S., Horányi, A., Muñoz‐Sabater, J., (42 named authors) & Thépaut, J. N, 2020: The ERA5 global reanalysis. Quarterly Journal of the Royal Meteorological Society, 146(730), 1999-2049, DOI: https://doi.org/10.1002/qj.3803

Immler, F. J., Dykema, J., Gardiner, T., Whiteman, D. N., Thorne, P. W., and Vömel, H., 2010: Reference Quality Upper-Air Measurements: guidance for developing GRUAN data products, Atmos. Meas. Tech., 3, 1217–1231, DOI:: https://doi.org/10.5194/amt-3-1217-2010

Karlsson, K.-G., Stengel, M., Meirink, J. F., Riihelä, A., Trentmann, J., Akkermans, T., Stein, D., Devasthale, A., Eliasson, S., Johansson, E., Håkansson, N., Solodovnik, I., Benas, N., Clerbaux, N., Selbach, N., Schröder, M., and Hollmann, R, 2023: CLARA-A3: The third edition of the AVHRR-based CM SAF climate data record on clouds, radiation and surface albedo covering the period 1979 to 2023, Earth Syst. Sci. Data, 15, 4901–4926, DOI: https://doi.org/10.5194/essd-15-4901-2023.

McGarragh, G.R.; Poulsen, C.A.; Thomas, G.E.; Povey, A.C.; Sus, O.; Stapelberg, S.; Schlundt, C.; Proud, S.; Christensen, M.W.; Stengel, M.; et al., 2018: The Community Cloud retrieval for CLimate (CC4CL). Part II: The optimal estimation approach. Atmos. Meas. Tech., 11, 3397–3431, DOI: https://doi.org/10.5194/amt-11-3397-2018, 2018

Mueller, R., C. Matsoukas, A. Gratzki, H. Behr, and R. Hollmann, 2009: The CM-SAF operational scheme for the satellite based retrieval of solar surface irradiance - A LUT based eigenvector hybrid approach, Remote Sensing of Environment, 113(5), 1012-1024, DOI: https://doi.org/10.1016/j.rse.2009.01.012

Ohmura, A., et al., 1998: Baseline Surface Radiation Network (BSRN/WCRP): New precision radiometry for climate research, Bulletin of the American Meteorological Society, 79(10), 2115-2136, DOI: https://doi.org/10.1175/1520-0477(1998)079<2115:BSRNBW>2.0.CO;2

Schaaf, C. and Wang, Z., 2015: MCD43C1 MODIS/Terra+Aqua BRDF/AlbedoModel Parameters Daily L3 Global 0.05Deg CMG V006, NASA EOSDIS Land Processes DAAC, DOI: https://doi.org/10.5067/MODIS/MCD43C1.006.

Schwarz, M., Folini, D., Hakuba, M. Z., & Wild, M., 2017: Spatial representativeness of surface-measured variations of downward solar radiation. Journal of Geophysical Research: Atmospheres, 122, 13,319– 13,337. DOI: https://doi.org/10.1002/2017JD027261

Smith D. 2012: ENVISAT AATSR Instrument Performance – End of Mission Report. Report, accessed 27-May-2024: https://atsrsensors.org/pdf/ENVISAT%20AATSR%20Instrument%20Performance%20-%20End%20of%20Mission%20Report%20%20-%20Issue%201.0.pdf

 Stoffelen, A., 1998: Towards the true near-surface wind speed: Error modeling and calibration using triple collocation. J. Geophys. Res. 103C3, 7755-7766, DOI: https://doi.org/10.1029/97JC03180

Sus, O.; Stengel, M.; Stapelberg, S.; McGarragh, G.; Poulsen, C.; Povey, A.C.; Schlundt, C.; Thomas, G.; Christensen, M.; Proud, S.; et al., 2018: The Community Cloud retrieval for Climate (CC4CL). Part I: A framework applied to multiple satellite imaging sensors. Atmos. Meas. Tech., 11, 3373–3396, DOI: https://doi.org/10.5194/amt-11-3373-2018

Earth Resources Observation and Science Center/U.S. Geological Survey/U.S. Department of the Interior. 1997. USGS 30 ARC-second Global Elevation Data, GTOPO30. Research Data Archive at the National Center for Atmospheric Research, Computational and Information Systems Laboratory. DOI: https://doi.org/10.5065/A1Z4-EE71 (https://rda.ucar.edu/datasets/ds758.0/#)

Wielicki, B. A., Barkstrom, B. R., Harrison, E. F., Lee III, R. B., Smith, G. L., & Cooper, J. E., 1996: Clouds and the Earth's Radiant Energy System (CERES): An earth observing system experiment. Bulletin of the American Meteorological Society, 77(5), 853-868, DOI: https://doi.org/10.1175/1520-0477(1996)077%3C0853:CATERE%3E2.0.CO;2


This document has been produced in the context of the Copernicus Climate Change Service (C3S).

The activities leading to these results have been contracted by the European Centre for Medium-Range Weather Forecasts, operator of C3S on behalf of the European Union (Delegation Agreement signed on 11/11/2014 and Contribution Agreement signed on 22/07/2021). All information in this document is provided "as is" and no guarantee or warranty is given that the information is fit for any particular purpose.

The users thereof use the information at their sole risk and liability. For the avoidance of all doubt , the European Commission and the European Centre for Medium - Range Weather Forecasts have no liability in respect of this document, which is merely representing the author's view.

Related articles