Last modified on Jun 21, 2023 18:40

Contributors: E. Carboni (UKRI-STFC RAL Space), G.E. Thomas (UKRI-STFC RAL Space)

Issued by: STFC RAL Space (UKRI-STFC) / Elisa Carboni

Date: 27/04/2023

Ref: C3S2_D312a_Lot1.2.3.5-v4.0_202304_PQAR_CCISurfaceRadiationBudget_v1.1

Official reference number service contract: 2021/C3S2_312a_Lot1_DWD/SC1

History of modifications

List of datasets covered by this document

Related documents

Acronyms

List of tables

List of figures

General definitions

The “CCI product family” Climate Data Record (CDR) consists of two parts. The ATSR2-AATSR Surface Radiation Budget CDR is formed by a TCDR brokered from the ESA Cloud_cci project and an ICDR derived from the Sea and Land Surface Temperature Radiometer (SLSTR) on board of Sentinel-3A and -B. ICDR uses the same processing and infrastructure as the TCDR. Both TCDR and ICDR data have been produced by STFC RAL Space.

These Surface Radiation Budget datasets from polar orbiting satellites consists of seven main variables: Surface Incoming Shortwave radiation (SIS), Surface Reflected Shortwave radiation (SRS), the Surface Net Shortwave radiation (SNS), the Surface Outgoing Longwave radiation (SOL), Surface Downwelling Longwave radiation (SDL), Surface Net Longwave radiation (SNL), and the Surface Radiation Budget (SRB).

Bias (accuracy): Mean difference between TCDR/ICDR and reference data

\( b=\frac{\sum_{i=1}^N (p_i - r_i)}{N} \ \ (Eq. 1) \)

Where: p_i is the CDR product, b is the mean bias and r_i is the equivalent value from the reference dataset. N is the number of observations.

bc-RMSE (precision): Bias corrected root mean squared error to express the precision of TCDR/ICDR compared to a reference data record

\( bc- RMSE=\sqrt{\frac{\sum_{i=1}^N ((p-b)-r)^2}{N}} \ \ (Eq. 2) \)

Where: p_i is the CDR product, b is the mean bias and r_i is the equivalent value from the reference dataset. N is the number of observations.

Stability: The variation of the bias over a multi-annual time period

Table 1: Summary of variables and definitions

Table 2: Definition of processing levels

Table 3: Definition of various technical terms used in the document

Scope of the document

This document provides a description of the product validation results for the Climate Data Record (CDR) of the Essential Climate Variable (ECV) Surface Radiation Budget. This CDR comprises inputs from two sources: (i) brokered products from the Cloud Climate Change Initiative (ESA’s Cloud_cci), namely those coming from processing of the Advanced Along-Track Scanning Radiometer (A)ATSR) data and (ii) those produced under this contract for the Climate Data Store, specifically those coming from processing of the Sea and Land Surface Temperature Radiometers (SLSTR).

The Thematic Climate Data Record (TCDR) is the product brokered from the European Space Agency Cloud Climate Change Initiative (ESA’s Cloud_cci) ATSR2-AATSR version 3.0 (Level-3C) dataset. This is produced by STFC RAL Space from the second Along-Track Scanning Radiometer (ATSR-2) on board the second European Remote Sensing Satellite (ERS-2) spanning the period 1995-2003 and the Advanced ATSR (AATSR) on board ENVISAT, spanning the period 2002-2012.

In addition, the Interim Climate Data Record (ICDR) is the product derived from the SLSTR onboard Sentinel-3A and –B and spans the period from January 2017 to present. Validation of this SLSTR derived product for the period from January 2017 to March 2022 is described in this document.

This document summarizes and refers to the methodology presented in the Cloud_cci Product Validation and Intercomparison Report [D1], used for the validation of the TCDR product. The same methodology is applied to the ICDR dataset.

Executive Summary

The ESA Climate Change Initiative (CCI) Surface Radiation Budget Climate Data Record (CDR) is a brokered product from the ESA Cloud_cci project (TCDR), while the extension Interim CDR (ICDR), produced from the Sea and Land Surface Temperature Radiometer (SLSTR), is produced specifically for C3S. The product is generated by STFC RAL Space, using the Community Cloud for Climate (CC4CL) processor, based on the Optimal Retrieval of Aerosol and Cloud (ORAC) algorithm. The Surface Radiation Budget is a product of the Broadband Radiative Flux Retrieval (BRFR) module of CC4CL, which uses the cloud properties produced by ORAC to compute broadband radiative flux values. Please find further information in the Algorithm Theoretical Basis Document (ATBD) [D4].

The Cloud_cci dataset comprises 17 years (1995-2012) of satellite-based measurements derived from the Along Track Scanning Radiometers (ATSR-2 and AATSR) onboard the ESA second European Research Satellite (ERS-2) and ENVISAT satellites. This TCDR is partnered with the ICDR produced from the Sentinel-3A SLSTR, beginning in 2017, and Sentinel-3B SLSTR beginning in October 2018.

The dataset encompasses level-3 data (monthly means) on a regular global latitude-longitude grid (with a resolution of 0.5°´ 0.5°) and includes these products: the Surface Incoming and Reflected Shortwave radiation (SIS and SRS respectively), the Surface Downwelling and Outgoing Longwave radiation (SDL and SOL respectively), the Surface Net Shortwave and Longwave radiation (SNS and SNL), and the total Surface Radiation Budget (SRB). Table 2-1 provides a summary of the calculated accuracies of the Surface Radiation Budget dataset (see section 2).

This document is divided in different sections:

• the first section presents a brief description of validation methodology together with reference for further information;
• the second section presents the results of the validation and comparison of TCDR and ICDR data;
• the third section presents the compliance with user requirements and includes recommendation on the usage and know limitations.

1. Product validation methodology

The validation methodology is described in section 2.4 of [D1]. In summary, the methodology uses the bias between the Cloud_cci product and the reference data to estimate the accuracy of the dataset. The bias corrected root mean squared error (bc-RMSE) is used to express the precision of the CDR compared to a reference data record, which is also known as the standard deviation about the mean. Ground data from Baseline Surface Radiation Network (BSRN¹) stations are considered as a validation reference, and satellite estimates (e.g. the CERES surface radiation dataset) are considered for the comparison. Stability is calculated as the variation of the bias over a multi-annual time period. Table 1-1 summarizes the methodology used to estimate the accuracies for each product.

Table 1-1: Summary of methodologies used to estimate the accuracies, for TCDR and ICDR datasets.

The Product Validation and Intercomparison Report [D1] includes the validation and intercomparison of the TCDR Surface Radiation Budget versus the CERES satellite dataset. The same methodology is used for the ICDR.

2. Validation results

The validation results for the TCDR products are presented and described in detail in [D1], sections 3.3.2, 5.3 and 5.4. In this document, a summary highlighting the main results is presented.  Only SIS and SDL in the TCDR product are validated with BSRN.   All other properties (including SIS and SDL in the ICDR) are compared with CERES. The evaluation with CERES is considered to be a comparison because the CERES surface radiation dataset has a similar accuracy to the CDR dataset.

Table 2-1: Summary of the accuracy of the Surface Radiation Budget dataset. The ‘bold’ accuracies come from direct validation with a ground measurement network (BSRN), the others come from the intercomparison with similar datasets (CERES) or with an uncertainty propagation. ICDR values are obtained from data between January 2017 and December 2021 for SLSTR-A and between October 2018 and December 2021 for SLSTR-B.

 The ICDR data of SIS, SRS, SOL and SDL are compared with CERES, using the methodology described in [D1] section 5.3 and 5.4, and results are presented here in section 2.2.

There is no direct validation (e.g. against more accurate measurements) for the net fluxes and these accuracies are estimated by error propagation. Sections 2.4, 2.5, 2.6 in this document present the estimate of the accuracy for SNS, SNL and SRB. Table 2-1 provides a summary of the CDR accuracies.

2.1 Validation with BSRN ground base radiative flux

BSRN stations measure direct, diffuse and global downwelling shortwave and longwave fluxes with a 1-minute temporal resolution. The 1-minute data was aggregated to monthly averages which were used for the validation. Using the TCDR and the reference datasets (for multiple locations around the world (see Figures 2-1 and 2-2)) we compute the bias and standard deviation.

The validation for Surface Incoming Shortwave radiation (SIS) and Surface Downwelling Longwave radiation (SDL) with BSRN ground measurements is described in section 3.3.2 of [D1].

Validation of BOA fluxes (i.e. SIS and SDL) in the TCDR against BSRN stations result in a standard deviation of 24 W/m² and a bias of 8.2 W/m² for shortwave radiation (SIS) and a standard deviation of 14 W/m² and a bias of 11.9 W/m² for longwave radiation (SDL).

Figures 2-1 and 2-2 show the results of the TCDR comparison with the BSRN incoming shortwave (SIS) and longwave (SDL) radiation with scatter plots and global maps showing the bias for each station.

Figure 2-1: Results from [D1]; Top: Validation results for Cloud_cci surface incoming shortwave (SIS) flux using BSRN as a reference. This covers data for the period 01-2003 to 12-2016. Bottom: Bias for each ground station over the same period.

Figure 2-2: Results from [D1]; Top: Validation results for Cloud_cci surface downwelling longwave (SDL) flux using BSRN as a reference. This covers data for the period 01-2003 to 12-2016. Bottom: Bias for each ground station over the same period.

2.2 Comparison with CERES satellite data

The TCDR and reference datasets from CERES are compared by means of multi-annual mean and standard deviation, all for a common time period (2003-2011). Global maps of multiannual Surface Incoming Shortwave radiation (SIS) and Surface Downwelling Longwave radiation (SDL) are computed for the TCDR and reference dataset. The scores (bias and bc-RMSE) are calculated by including all valid data points (for a latitude band of 60°S - 60°N) pairwise in the CERES and the Cloud_cci dataset.

The validation for Surface Incoming Shortwave radiation (SIS) and Surface Downwelling Longwave radiation (SDL) with CERES is described in section 5.3 and 5.4 of [D1]. Intercomparison of Cloud_cci radiation products with CERES present bias of 1.53 W/m² for SIS and bias of 10.17 W/m² for SDL.

The stability estimated for the TCDR dataset are 0.97 and 2.76 W/m²/decade for SIS and SDL respectively. The dataset is relatively stable during this period but shows some variations in the downwelling longwave radiation.

The same methodology is used to estimate the accuracy of SOL in comparison with CERES. Intercomparison of Cloud_cci radiation products (TCDR) with CERES present bias of 11 W/m² for SOL, which is the accuracy value reported in Table 2-1.

The evaluation method is also used to estimate the SIS, SRS, SOL and SDL accuracy of the ICDR.

Figures 2-3 and 2-4 show an example of ICDR products for March 2017 and the equivalent monthly mean product from CERES.

Figure 2-3: SRS, SOL, SIS and SDL values from SLSTR (ICDR dataset) for March 2017.

Figure 2-4: SRS, SOL, SIS and SDL values from CERES dataset for March 2017.

Bias between TCDR and CERES are calculated including all valid data points (for a latitude band of 60° S-60° N). Intercomparison of ICDR radiation products with CERES has been performed using the data between January 2017 and March 2022. The resulting bias (reported in Table 2-1) are the following:

SLSTR-A: 1.8 W/m² for SIS, 1.6 W/m² for SRS, 1.6 W/m² for SOL and 9.7 W/m² for SDL

SLSTR-B: 0.23 W/m² for SIS, 2.1 W/m² for SRS, 4.1 W/m² for SOL, 11 W/m² for SDL 

SLSTR-A+B:  0.51 W/m² for SIS, 2.2 W/m² for SRS, 3.8 W/m² for SOL, 11 W/m² for SDL 

General findings:

SIS (from [D1] section 5.3)

• The CDR dataset shows very similar patterns to the other Cloud_cci datasets of the global mean bottom of the atmosphere incoming solar radiation. Larger values are found for the subtropics; the maximum is located in the Atacama Desert. Lowest mean BOA incoming solar radiation is found over the polar regions.
• Spread among all the Cloud_cci datasets is highest for polar regions and high latitudes. Also the stratocumulus regions are clearly noticeable in the dataset.
• CDR dataset compared to CERES present similar patterns of temporal variability. The tropics show the lowest variability over time, while polar landmasses show the highest. CERES EBAF-SURFACE Ed4.0 contains both the highest and lowest values for temporal variability, but CDR and the other Cloud_cci datasets are alike.
• For the period from 2003 to 2011 no significant trends or anomalies are determinable. CDR dataset is relatively stable during this period and only show small variations in the BOA incoming solar radiation.

SDL (from [D1] section 5.4):

• Multi-annual global means of CDR and all Cloud_cci datasets compare very well with each other and hardly any larger differences are visible. The highest BOA downwelling thermal radiation is found over the tropics and subtropics, lowest values are found in Antarctica.
• With the exception of the inner tropics, there are slight differences between sea and land BOA downwelling thermal radiation in all datasets. Higher values are measured over the ocean than over land. Over land, mountains such as the Himalayas or the Andes are noticeable due to their clearly lower values.
• Similar to the global mean, the temporal variability of CDR datasets also shows a good agreement. In addition, the variability over land is significantly higher than over the ocean, with the exception of a narrow band in the inner tropics. The highest variability is found in East Asia.
• Strong seasonal cycles are visible with higher values in boreal summer and lower values in boreal winter. CDR time series, as well as all Cloud_cci datasets, also contain a small positive trend.

2.3 Surface Reflected Shortwave Radiation (SRS)

The TCDR accuracy of SRS is estimated from the accuracy of the Surface Incoming Shortwave radiation (SIS) and the Surface albedo (SAL).

Applying the error propagation, the accuracy of the SRS product can be estimated as:

\( \Delta SRS= \frac{\delta SRS}{\delta SIS} \Delta SIS + \frac{\delta SRS}{\delta SAL} \Delta SAL = SAL \Delta SIS + SIS \Delta SAL, \quad \ \ (Eq. 3) \)

Where \( \Delta SIS \) comes from [D1] and DSAL is considered as 25% of the SAL value. SAL is estimated as the ratio between Surface Reflected Shortwave radiation (SRS) and Surface Incoming Shortwave radiation (SIS).

\( SAL = SRS / SIS, \quad \ \ (Eq. 4) \)

The resulting global mean accuracy for the TCDR SRS is 4.6 W/m².

Accuracies of ICDR SRS are 1.6, 2.1 and 2.2 W/m² for SLSTR-A, SLSTR-B and SLSTR A+B respectively and are estimated in comparison with CERES (section 2.2).

2.4 Surface Net Shortwave Radiation (SNS)

The Surface Net Shortwave radiation (SNS) is calculated using:

\( SNS = SIS - SRS, \quad \ \ (Eq. 5) \)

And the accuracy will be estimated as:

\( \Delta SNS = \Delta SIS + \Delta SRS, \quad \ \ (Eq. 6) \)

The resulting global mean accuracy for the SNS is 13 W/m² for TCDR, 3.4, 2.3 and 2.7 W/m² for ICDR SLSTR-A, SLSTR-B and SLSTR-A+B respectively.

2.5 Surface Net Longwave Radiation (SNL)

Surface Net Longwave radiation (SNL) is calculated [D2] from:

\( SNL = SDL - SOL, \quad \ \ (Eq. 7) \)

The accuracy \( \Delta SNL \)  will be estimated as:

\( \Delta SNL = \Delta SDL + \Delta SOL, \quad \ \ (Eq. 8) \)

The resulting global mean accuracy for the SNL is 23 W/m² for TCDR; 11 W/m² , 15 W/m² and  15 W/m² for ICDR SLSTR-A , B and A+B respectively.

2.6 Surface Radiation Budget (SRB)

The total Surface Radiation Budget (SRB) is the sum of the short and longwave contributions:

\( SRB = SNS + SNL, \quad \ \ (Eq. 9) \)

The accuracy will be estimated as:

\( \Delta SRB = \Delta SNS + \Delta SNL, \quad \ \ (Eq. 10) \)

The resulting global mean accuracy for the SRB is 36 W/m² for TCDR, 14 W/m² , 17 W/m² and 18 W/m² for ICDR SLSTR-A , B and A+B respectively.

3. Application(s) specific assessment

This section is not applicable. There are no additional application specific assessments known since the dataset has just been published.

4. Compliance with user requirements

There are no direct user requirements for the Surface Radiation Budget defined in the Cloud_cci project. Looking at the GCOS ECV requirements for Surface Radiation Budget² the values for SIS and SDL are 1 W/m² uncertainty, while the TCDR dataset achieves an accuracy of 8 W/m² for SIS and 12 W/m² for SDL therefore they currently do not meet the GCOS requirements. Please find more detailed information about the target requirements in the corresponding (Target Requirement and Gap Analysis Document) TRGAD [D3].

ICDR accuracies (estimated with the first 5 years (2017 -2021)) appear to be better than TCDR accuracies. ICDR accuracies have been estimated in comparison with CERES surface fluxes.  The better agreement of the ICDR dataset with CERES could come from different factors: (i) comparison of satellite vs satellite instead of satellite vs ground measurements; (ii) wider swath of SLSTR measurements; (iii) closest assumption (auxiliary data) used by both CERES and C3S estimate in the period considered (SRS and SOL strongly depend on surface reflectance and emissivity).  

Table 4-1 provides an overview of the GCOS requirements for the surface radiative balance and the values achieved by the TCDR parameters. ICDR accuracies, estimated with 5 years of comparison with a similar satellite dataset, show results consistent with TCDR. It should be noted that GCOS requirements are targets and are often not attainable using existing or historical observing systems. The Cloud_cci doesn’t meet the requirement for resolving the diurnal cycle due to the nature of the satellite observations, but exceeds the spatial resolution.

Table 4-1: GCOS targets for Earth Radiation Budget ECVs and CDR values. TCDR values taken from Table 5-4 and Table 5-5 in [D1].

 Known limitations [From D1 table 7.1]:

• Higher uncertainties in twilight conditions, especially in the shortwave fluxes, due to limitation in retrieving cloud optical thickness and cloud particle effective radius (input to the radiation calculation) in these conditions
• Partly sparse temporal/spatial sampling (partly compensated by introduced diurnal cycles correction).
• Downwelling longwave fluxes seem biased high.

References

n/a

Related articles

• Page:
  ECV - Glaciers, Ice Sheets, Lakes, Soil Moisture
• Page:
  IV Version 1.2; SEC Version 2.0; GMB Version 2.0: Product User Guide Specification (PUGS)
• Page:
  IV Version 1.2; SEC Version 2.0; GMB Version 2.0: System Quality Assurance Document (SQAD)
• Page:
  IV Version 1.2; SEC Version 2.0; GMB Version 2.0: Product Quality Assurance Document (PQAD)
• Page:
  IV Version 1.2; SEC Version 2.0; GMB Version 2.0: Product Quality Assessment Report (PQAR)