Page info infoType Modified date prefix Last modified on type Flat
Contributors: E. Carboni (UKRI-STFC RAL Space), G.E. Thomas (UKRI-STFC RAL Space)
Issued by: STFC RAL Space (UKRI-STFC) / Elisa Carboni
Date: 1027/0304/20212023
Ref: C3SC3S2_D312bD312a_Lot1.2.23.65-v3v4.20_202103202304_PQAR_CCISurfaceRadiationBudget_v1.01
Official reference number service contract: 20182021/C3SC3S2_312b312a_Lot1_DWD/SC1
Info | ||||
---|---|---|---|---|
| ||||
|
Easy Heading Macro | ||
---|---|---|
|
History of modifications
Expand | ||||||||
---|---|---|---|---|---|---|---|---|
| ||||||||
|
List of datasets covered by this document
|
List of datasets covered by this document
Expand | |||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| |||||||||||||||
|
...
|
Anchor relateddocuments relateddocuments
Related documents
relateddocuments | |
relateddocuments |
Expand |
---|
...
Expand | ||||||||
---|---|---|---|---|---|---|---|---|
| ||||||||
|
Acronyms
|
Acronyms
Expand | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Expand | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Scope of the document
This document provides a description of the product validation results for the Essential Climate Variable (ECV) Surface Radiation Budget. These products are brokered to (in case of (A)ATSR) or produced for the Climate Data Store (in the case of SLSTR) by the Copernicus Climate Change Service (C3S).
The TCDR is a brokered version of ESA's Cloud_cci ATSR2-AATSR version 3.0 (Level-3C) dataset, produced by STFC RAL Space from the second Along-Track Scanning Radiometer (ATSR-2) onboard the second European Remote Sensing Satellite (ERS-2) spanning 1995-2003 and the Advanced ATSR (AATSR) onboard ENVISAT, which spanned 2002-2012. The ICDR is produced by C3S, using SLSTR onboard of Sentinel-3 and spans January 2017 to present (with a validation period from January/2017 – December/2018 in this version of the document).
Executive Summary
List of tables
Expand | ||
---|---|---|
| ||
Table 1-1: Summary of methodologies used to estimate the accuracies, for TCDR and ICDR datasets Table 2-1: Summary of the accuracy of the Surface Radiation Budget dataset Table 3-1: Summary of KPI results with 2.5 and 97.5 percentiles and number of ICDR months within the range Table 4-1: GCOS targets for Earth Radiation Budget ECVs and CDR values |
List of figures
Expand | ||
---|---|---|
| ||
Figure 2-1: Results from [D1]; Top: Validation results for Cloud_cci surface incoming shortwave (SIS) flux using BSRN as a reference Figure 2-2: Results from [D1]; Top: Validation results for Cloud_cci surface downwelling longwave (SDL) flux using BSRN as a reference Figure 2-3: SRS, SOL, SIS and SDL values from SLSTR (ICDR dataset) for March 2017 Figure 2-4: SRS, SOL, SIS and SDL values from CERES dataset for March 2017 |
General definitions
The “CCI product family” Climate Data Record (CDR) consists of two parts. The ATSR2-AATSR Surface Radiation Budget CDR is formed by a TCDR brokered from the ESA Cloud_cci project and an ICDR derived The ESA Climate Change Initiative (CCI) Surface Radiation Budget Climate Data Record (CDR) is a brokered product from the ESA Cloud_cci project, while the extension Interim CDR (ICDR), produced from the Sea and Land Surface Temperature Radiometer (SLSTR) , is produced specifically for C3S. The product is generated by STFC RAL Space, using the Community Cloud for Climate (CC4CL) processor, based on the Optimal Retrieval of Aerosol and Cloud (ORAC) algorithm. The Surface Radiation Budget is a product of the Broadband Radiative Flux Retrieval (BRFR) module of CC4CL, which uses the cloud properties produced by ORAC to compute broadband radiative flux values.
The Cloud_cci dataset comprises 17 years (1995-2012) of satellite-based measurements derived from the Along Track Scanning Radiometers (ATSR-2 and AATSR) onboard the ESA second European Research Satellite (ERS-2) and ENVISAT satellites. This TCDR is partnered with the ICDR produced from the Sentinel-3A SLSTR, beginning in 2017, and Sentinel-3B SLSTR beginning in October 2018. The dataset encompasses level-3 data (monthly means) on a regular global latitude-longitude grid (with a resolution of 0.5° × 0.5°) and includes these products: the Surface Incoming and Reflected Shortwave radiation (SIS and SRS respectively), the Surface Downwelling and Outgoing Longwave radiation (SDL and SOL respectively), the Surface Net Shortwave and Longwave radiation (SNS and SNL), and the total Surface Radiation Budget (SRB).
Table 1 provides a summary of the calculated accuracies.
...
on board of Sentinel-3A and -B. ICDR uses the same processing and infrastructure as the TCDR. Both TCDR and ICDR data have been produced by STFC RAL Space.
These Surface Radiation Budget datasets from polar orbiting satellites consists of seven main variables: Surface Incoming Shortwave radiation (SIS), Surface Reflected Shortwave radiation (SRS), the Surface Net Shortwave radiation (SNS), the Surface Outgoing Longwave radiation (SOL), Surface Downwelling Longwave radiation (SDL), Surface Net Longwave radiation (SNL), and the Surface Radiation Budget (SRB).
Bias (accuracy): Mean difference between TCDR/ICDR and reference data
Mathinline |
---|
b=\frac{\sum_{i=1}^N (p_i - r_i)}{N} \ \ (Eq. 1) |
Where: pi is the CDR product, b is the mean bias and ri is the equivalent value from the reference dataset. N is the number of observations.
bc-RMSE (precision): Bias corrected root mean squared error to express the precision of TCDR/ICDR compared to a reference data record
Mathinline |
---|
bc- RMSE=\sqrt{\frac{\sum_{i=1}^N ((p-b)-r)^2}{N}} \ \ (Eq. 2) |
Where: pi is the CDR product, b is the mean bias and ri is the equivalent value from the reference dataset. N is the number of observations.
Stability: The variation of the bias over a multi-annual time period
Table 1: Summary of variables and definitions Anchor table1 table1
Variables | Abbreviation | Definition |
Surface incoming solar radiation | SIS | The total incoming solar flux, measured at the Earth’s surface. |
Surface reflected solar radiation | SRS | The total upwelling shortwave flux, measured at the Earth’s surface. |
Surface net solar radiation | SNS | The net downwelling solar flux, measured at the surface (equal to SIS – SRS). |
Surface downwelling longwave radiation | SDL
| The total downwelling thermal infrared flux, measured at the Earth’s surface. |
Surface outgoing longwave radiation
| SOL
| The total upwelling thermal infrared flux, measured at the Earth’s surface. |
Surface net longwave radiation | SNL | The net downwelling thermal infrared flux, measured at the Earth’s surface (equal to SDL-SOL). |
Total surface radiation budget | SRB | The total net downwelling radiative flux, measured at the Earth’s surface (equal to (SIS+SDL) – (SRS+SOL)). |
Table 2: Definition of processing levels Anchor table2 table2
Processing level | Definition |
Level-1b | The full-resolution geolocated radiometric measurements (for each view and each channel), rebinned onto a regular spatial grid. |
Level-2 (L2) | Retrieved cloud variables at full input data resolution, thus with the same resolution and location as the sensor measurements (Level-1b). |
Level-3C (L3C) | Cloud properties of Level-2 orbits of one single sensor combined (averaged) on a global spatial grid. Both daily and monthly products provided through C3S are Level-3C. |
Table 3: Definition of various technical terms used in the document Anchor table3 table3
Jargon | Definition |
Brokered product | The C3S Climate Data Store (CDS) provides both data produced specifically for C3S and so-called brokered products. The latter are existing products produced under an independent programme or project which are made available through the CDS. |
Climate Data Store (CDS) | The front-end and delivery mechanism for data made available through C3S. |
Retrieval | A numerical data analysis scheme which uses some form of mathematical inversion to derive physical properties from some form of measurement. In this case, the derivation of cloud properties from satellite measured radiances. |
Forward model | A deterministic model which predicts the measurements made of a system, given its physical properties. The forward model is the function which is mathematically inverted by a retrieval scheme. In this case, the forward model predicts the radiances measured by a satellite instrument as a function of atmospheric and surface state, and cloud properties. |
TCDR | It is a consistently-processed time series of a geophysical variable of sufficient length and quality. |
ICDR | An Interim Climate Data Record (ICDR) denotes an extension of TCDR, processed with a processing system as consistent as possible to the generation of TCDR. |
CDR | A Climate Data Record (CDR) is defined as a time series of measurements with sufficient length, consistency, and continuity to determine climate variability and change. |
Scope of the document
This document provides a description of the product validation results for the Climate Data Record (CDR) of the Essential Climate Variable (ECV) Surface Radiation Budget. This CDR comprises inputs from two sources: (i) brokered products from the Cloud Climate Change Initiative (ESA’s Cloud_cci), namely those coming from processing of the Advanced Along-Track Scanning Radiometer (A)ATSR) data and (ii) those produced under this contract for the Climate Data Store, specifically those coming from processing of the Sea and Land Surface Temperature Radiometers (SLSTR).
The Thematic Climate Data Record (TCDR) is the product brokered from the European Space Agency Cloud Climate Change Initiative (ESA’s Cloud_cci) ATSR2-AATSR version 3.0 (Level-3C) dataset. This is produced by STFC RAL Space from the second Along-Track Scanning Radiometer (ATSR-2) on board the second European Remote Sensing Satellite (ERS-2) spanning the period 1995-2003 and the Advanced ATSR (AATSR) on board ENVISAT, spanning the period 2002-2012.
In addition, the Interim Climate Data Record (ICDR) is the product derived from the SLSTR onboard Sentinel-3A and –B and spans the period from January 2017 to present. Validation of this SLSTR derived product for the period from January 2017 to March 2022 is described in this document.
This document summarizes and refers to the methodology presented in the Cloud_cci Product Validation and Intercomparison Report [D1], used for the validation of the TCDR product. The same methodology is applied to the ICDR dataset.
Executive Summary
The ESA Climate Change Initiative (CCI) Surface Radiation Budget Climate Data Record (CDR) is a brokered product from the ESA Cloud_cci project (TCDR), while the extension Interim CDR (ICDR), produced from the Sea and Land Surface Temperature Radiometer (SLSTR), is produced specifically for C3S. The product is generated by STFC RAL Space, using the Community Cloud for Climate (CC4CL) processor, based on the Optimal Retrieval of Aerosol and Cloud (ORAC) algorithm. The Surface Radiation Budget is a product of the Broadband Radiative Flux Retrieval (BRFR) module of CC4CL, which uses the cloud properties produced by ORAC to compute broadband radiative flux values. Please find further information in the Algorithm Theoretical Basis Document (ATBD) [D4].
The Cloud_cci dataset comprises 17 years (1995-2012) of satellite-based measurements derived from the Along Track Scanning Radiometers (ATSR-2 and AATSR) onboard the ESA second European Research Satellite (ERS-2) and ENVISAT satellites. This TCDR is partnered with the ICDR produced from the Sentinel-3A SLSTR, beginning in 2017, and Sentinel-3B SLSTR beginning in October 2018.
The dataset encompasses level-3 data (monthly means) on a regular global latitude-longitude grid (with a resolution of 0.5°´ 0.5°) and includes these products: the Surface Incoming and Reflected Shortwave radiation (SIS and SRS respectively), the Surface Downwelling and Outgoing Longwave radiation (SDL and SOL respectively), the Surface Net Shortwave and Longwave radiation (SNS and SNL), and the total Surface Radiation Budget (SRB). Table 2-1 provides a summary of the calculated accuracies of the Surface Radiation Budget dataset (see section 2).
This document is divided in different sections:
- the first section presents a brief description of validation methodology together with reference for further information;
- the second section presents the results of the validation and comparison of TCDR and ICDR data;
- the third section presents the compliance with user requirements and includes recommendation on the usage and know limitations.
1. Product validation methodology
The validation methodology is described in section 2.4 of [D1]. In summary, the methodology uses the bias between the Cloud_cci product and the reference data to estimate the accuracy of the dataset.The bias corrected root mean squared error (bc-RMSE) is used to express the precision of the CDR compared to a reference data record, which is also known as the standard deviation about the mean. Ground data from Baseline Surface Radiation Network (BSRN1) stations are considered as a validation reference, and satellite estimates (e.g. the CERES surface radiation dataset) are considered for the comparison. Stability is calculated as the variation of the bias over a multi-annual time period. Table 1-1 summarizes the methodology used to estimate the accuracies for each product.
Table 1-1: Summary of methodologies used to estimate the accuracies, for TCDR and ICDR datasets Anchor table1_1 table1_1
Product name | Validation with BSRN | Comparison with CERES | Uncertainty propagation |
Surface Incoming Shortwave radiation (SIS) | TCDR | TCDR and ICDR |
|
Surface Reflected Shortwave radiation (SRS) |
| TCDR and ICDR | TCDR and ICDR |
Surface Net Shortwave radiation (SNS) |
|
| TCDR and ICDR |
Surface Outgoing Longwave radiation (SOL) |
| TCDR and ICDR |
|
Surface Downwelling Longwave radiation (SDL) | TCDR | TCDR and ICDR |
|
Surface Net Longwave radiation (SNL) | TCDR and ICDR | ||
Surface Radiation Budget (SRB) |
|
| TCDR and ICDR |
The Product Validation and Intercomparison Report [D1] includes the validation and intercomparison of the TCDR Surface Radiation Budget versus the CERES satellite dataset. The same methodology is used for the ICDR.
Info | ||
---|---|---|
| ||
1 Please find more information on the reference data on [D1]: BSRN (Annex A.5) and CERES (A.6) |
Anchor | ||||
---|---|---|---|---|
|
The validation results for the TCDR products are presented and described in detail in [D1], sections 3.3.2, 5.3 and 5.4. In this document, a summary highlighting the main results is presented. Only SIS and SDL in the TCDR product are validated with BSRN. All other properties (including SIS and SDL in the ICDR) are compared with CERES. The evaluation with CERES is considered to be a comparison because the CERES surface radiation dataset has a similar accuracy to the CDR dataset.
Table 2-1: Summary of the accuracy of the Surface Radiation Budget dataset. The ‘bold’ accuracies come from direct validation with a ground measurement network (BSRN), the others come from the intercomparison with similar datasets (CERES) or with an uncertainty propagation. ICDR values are obtained from data between January 2017 and December 2021 for SLSTR-A and between October 2018 and December 2021 for SLSTR-B. Anchor table2_1 table2_1
Product name | TCDR Accuracy [W/m2] | ICDR SLSTR-A Accuracy [W/m2] | ICDR SLSTR-B Accuracy [W/m2] | ICDR A+B Accuracy [W/m2] |
Surface Incoming Shortwave radiation (SIS) | 8.2 | 1.8 | 0.23 | 0.51 |
Surface Reflected Shortwave radiation (SRS) | 4.6 | 1.6 | 2.1 | 2.2 |
Surface Net Shortwave radiation (SNS) | 13 | 3.4 | 2.3 | 2.7 |
Surface Outgoing Longwave radiation (SOL) | 11 | 1.6 | 4.1 | 3.8 |
Surface Downwelling Longwave radiation (SDL) | 12 | 9.7 | 11 | 11 |
Surface Net Longwave radiation (SNL) | 23 | 11 | 15 | 15 |
Surface Radiation Budget (SRB) | 36 | 14 | 17 | 18 |
The ICDR data of SIS, SRS, SOL and SDL are compared with CERES, using the methodology described in [D1] section 5.3 and 5.4, and results are presented here in section 2.2.
There is no direct validation (e.g. against more accurate measurements) for the net fluxes and these accuracies are estimated by error propagation. Sections 2.4, 2.5, 2.6 in this document present the estimate of the accuracy for SNS, SNL and SRB. Table 2-1 provides a summary of the CDR accuracies.
2.1 Validation with BSRN ground base radiative flux
BSRN stations measure direct, diffuse and global downwelling shortwave and longwave fluxes with a 1-minute temporal resolution. The 1-minute data was aggregated to monthly averages which were used for the validation. Using the TCDR and the reference datasets (for multiple locations around the world (see Figures 2-1 and 2-2)) we compute the bias and standard deviation.
The validation for Surface Incoming Shortwave radiation (SIS) and Surface Downwelling Longwave radiation (SDL) with BSRN ground measurements is described in section 3.3.2 of [D1].
Validation of BOA fluxes (i.e. SIS and SDL) in the TCDR against BSRN stations result in a standard deviation of 24 W/m² and a bias of 8.2 W/m² for shortwave radiation (SIS) and a standard deviation of 14 W/m² and a bias of 11.9 W/m² for longwave radiation (SDL).
Figures 2-1 and 2-2 show the results of the TCDR comparison with the BSRN incoming shortwave (SIS) and longwave (SDL) radiation with scatter plots and global maps showing the bias for each station.
Figure 2-1: Results from [D1]; Top: Validation results for Cloud_cci surface incoming shortwave (SIS) flux using BSRN as a reference. This covers data for the period 01-2003 to 12-2016. Bottom: Bias for each ground station over the same period. Anchor figure2_1 figure2_1
Figure 2-2: Results from [D1]; Top: Validation results for Cloud_cci surface downwelling longwave (SDL) flux using BSRN as a reference. This covers data for the period 01-2003 to 12-2016. Bottom: Bias for each ground station over the same period. Anchor figure2_2 figure2_2
2.2 Comparison with CERES satellite data
The TCDR and reference datasets from CERES are compared by means of multi-annual mean and standard deviation, all for a common time period (2003-2011). Global maps of multiannual Surface Incoming Shortwave radiation (SIS) and Surface Downwelling Longwave radiation (SDL) are computed for the TCDR and reference dataset. The scores (bias and bc-RMSE) are calculated by including all valid data points (for a latitude band of 60°S - 60°N) pairwise in the CERES and the Cloud_cci dataset.
The validation for Surface Incoming Shortwave radiation (SIS) and Surface Downwelling Longwave radiation (SDL) with CERES is described in section 5.3 and 5.4 of [D1]. Intercomparison of Cloud_cci radiation products with CERES present bias of 1.53 W/m² for SIS and bias of 10.17 W/m2 for SDL.
The stability estimated for the TCDR dataset are 0.97 and 2.76 W/m²/decade for SIS and SDL respectively. The dataset is relatively stable during this period but shows some variations in the downwelling longwave radiation.
The same methodology is used to estimate the accuracy of SOL in comparison with CERES. Intercomparison of Cloud_cci radiation products (TCDR) with CERES present bias of 11 W/m² for SOL, which is the accuracy value reported in Table 2-1.
The evaluation method is also used to estimate the SIS, SRS, SOL and SDL accuracy of the ICDR.
Figures 2-3 and 2-4 show an example of ICDR products for March 2017 and the equivalent monthly mean product from CERES.
Figure 2-3: SRS, SOL, SIS and SDL values from SLSTR (ICDR dataset) for March 2017. Anchor figure2_3 figure2_3
Figure 2-4: SRS, SOL, SIS and SDL values from CERES dataset for March 2017. Anchor figure2_4 figure2_4
Bias between TCDR and CERES are calculated including all valid data points (for a latitude band of 60° S-60° N). Intercomparison of ICDR radiation products with CERES has been performed using the data between January 2017 and March 2022. The resulting bias (reported in Table 2-1) are the following:
SLSTR-A: 1.8 W/m² for SIS, 1.6 W/m2 for SRS, 1.6 W/m2 for SOL and 9.7 W/m2 for SDL
SLSTR-B: 0.23 W/m² for SIS, 2.1 W/m² for SRS, 4.1 W/m² for SOL, 11 W/m² for SDL
SLSTR-A+B: 0.51 W/m² for SIS, 2.2 W/m² for SRS, 3.8 W/m² for SOL, 11 W/m² for SDL
General findings:
SIS (from [D1] section 5.3)
- The CDR dataset shows very similar patterns to the other Cloud_cci datasets of the global mean bottom of the atmosphere incoming solar radiation. Larger values are found for the subtropics; the maximum is located in the Atacama Desert. Lowest mean BOA incoming solar radiation is found over the polar regions.
- Spread among all the Cloud_cci datasets is highest for polar regions and high latitudes. Also the stratocumulus regions are clearly noticeable in the dataset.
- CDR dataset compared to CERES present similar patterns of temporal variability. The tropics show the lowest variability over time, while polar landmasses show the highest. CERES EBAF-SURFACE Ed4.0 contains both the highest and lowest values for temporal variability, but CDR and the other Cloud_cci datasets are alike.
- For the period from 2003 to 2011 no significant trends or anomalies are determinable. CDR dataset is relatively stable during this period and only show small variations in the BOA incoming solar radiation.
SDL (from [D1] section 5.4):
- Multi-annual global means of CDR and all Cloud_cci datasets compare very well with each other and hardly any larger differences are visible. The highest BOA downwelling thermal radiation is found over the tropics and subtropics, lowest values are found in Antarctica.
- With the exception of the inner tropics, there are slight differences between sea and land BOA downwelling thermal radiation in all datasets. Higher values are measured over the ocean than over land. Over land, mountains such as the Himalayas or the Andes are noticeable due to their clearly lower values.
- Similar to the global mean, the temporal variability of CDR datasets also shows a good agreement. In addition, the variability over land is significantly higher than over the ocean, with the exception of a narrow band in the inner tropics. The highest variability is found in East Asia.
- Strong seasonal cycles are visible with higher values in boreal summer and lower values in boreal winter. CDR time series, as well as all Cloud_cci datasets, also contain a small positive trend.
2.3 Surface Reflected Shortwave Radiation (SRS)
The TCDR accuracy of SRS is estimated from the accuracy of the Surface Incoming Shortwave radiation (SIS) and the Surface albedo (SAL).
Applying the error propagation, the accuracy of the SRS product can be estimated as:
Mathinline |
---|
\Delta SRS= \frac{\delta SRS}{\delta SIS} \Delta SIS + \frac{\delta SRS}{\delta SAL} \Delta SAL = SAL \Delta SIS + SIS \Delta SAL, \quad \ \ (Eq. 3) |
Where
Mathinline |
---|
\Delta SIS |
comes from [D1] and DSAL is considered as 25% of the SAL value. SAL is estimated as the ratio between Surface Reflected Shortwave radiation (SRS) and Surface Incoming Shortwave radiation (SIS).
Mathinline |
---|
SAL = SRS / SIS, \quad \ \ (Eq. 4) |
The resulting global mean accuracy for the TCDR SRS is 4.6 W/m2.
Accuracies of ICDR SRS are 1.6, 2.1 and 2.2 W/m2 for SLSTR-A, SLSTR-B and SLSTR A+B respectively and are estimated in comparison with CERES (section 2.2).
2.4 Surface Net Shortwave Radiation (SNS)
The Surface Net Shortwave radiation (SNS) is calculated using:
Mathinline |
---|
SNS = SIS - SRS, \quad \ \ (Eq. 5) |
And the accuracy will be estimated as:
Mathinline |
---|
\Delta SNS = \Delta SIS + \Delta SRS, \quad \ \ (Eq. 6) |
The resulting global mean accuracy for the SNS is 13 W/m2 for TCDR, 3.4, 2.3 and 2.7 W/m2 for ICDR SLSTR-A, SLSTR-B and SLSTR-A+B respectively.
2.5 Surface Net Longwave Radiation (SNL)
Surface Net Longwave radiation (SNL) is calculated [D2] from:
Mathinline |
---|
SNL = SDL - SOL, \quad \ \ (Eq. 7) |
The accuracy
Mathinline |
---|
\Delta SNL |
will be estimated as:
Mathinline |
---|
\Delta SNL = \Delta SDL + \Delta SOL, \quad \ \ (Eq. 8) |
The resulting global mean accuracy for the SNL is 23 W/m2 for TCDR; 11 W/m2 , 15 W/m2 and 15 W/m2 for ICDR SLSTR-A , B and A+B respectively.
2.6 Surface Radiation Budget (SRB)
The total Surface Radiation Budget (SRB) is the sum of the short and longwave contributions:
Mathinline |
---|
SRB = SNS + SNL, \quad \ \ (Eq. 9) |
The accuracy will be estimated as:
Mathinline |
---|
\Delta SRB = \Delta SNS + \Delta SNL, \quad \ \ (Eq. 10) |
The resulting global mean accuracy for the SRB is 36 W/m2 for TCDR, 14 W/m2 , 17 W/m2 and 18 W/m2 for ICDR SLSTR-A , B and A+B respectively.
3. Application(s) specific assessments
In addition to the extensive product validation (see chapter 2 for results and chapter 2/3 in [D6] for validation methodology) a second assessment is introduced to evaluate the Interim Climate Data Record (ICDR) against the Thematic Climate Data Record (TCDR) in terms of consistency. Since frequent ICDR deliveries make detailed validation not feasible, a consistency check against the deeply validated TCDR is used as an indication of quality. This is done by a comparison of the following two evaluations:
- TCDR against a stable, long-term and independent reference dataset
- ICDR against the same stable, long-term and independent reference dataset
The evaluation method is generated to detect differences in the ICDR performance in a quantitative, binary way with so called Key Performance Indicators. The general method is outlined in [D5] chapter 3. The same difference between TCDR/ICDR and the reference dataset would lead to the conclusion that TCDR and ICDR have the same quality (key performance is "good"). Variations or trends in the differences (TCDR/ICDR against reference) would require a further investigation to analyze the reasons. The key performance would be marked as "bad". The binary decision whether the key performance is good or bad is made in a statistical way by a hypotheses test (binomial test). Based on the TCDR/reference comparison (global means, monthly or daily means) a range is defined with 95% of the differences are within. This range (2.5 and 97.5 percentile) is used for the ICDR/reference comparison to check whether the values are in or out of the range. The results could be the following:
- All or a sufficient high number of ICDR/reference differences lies within the range defined by the TCDR/reference comparison: Key performance of the ICDR is "good"
- A smaller number of ICDR/reference differences is within the pre-defined range: Key performance of the ICDR is "bad"
3.1 Results
The results of the KPI test are summarized in Table 3-1.
Anchor | ||||
---|---|---|---|---|
|
Surface Incoming Shortwave Radiation | Surface Reflected Shortwave Radiation | Surface Outgoing Longwave Radiation | Surface Downwelling Longwave Radiation | ||
---|---|---|---|---|---|
Percentiles | p2.5 p97.5 | -1.29 W/m² 2.12 W/m² | -0.45 W/m² 0.36 W/m² | -16.4 W/m² 11.3 W/m² | -4.65 W/m² 4.48 W/m² |
Sentinel-3A: | |||||
01/2017 - 12/2020 | 11/37 | 08/37 | 00/37 | 35/37 | |
01/2017 - 12/2021 | 44/60 | 48/60 | 60/60 | 15/60 | |
01/2017 - 06/2022 | 44/63 | 51/63 | 63/63 | 15/63 | |
Sentinel-3B: | |||||
10/2018 - 12/2021 | 24/39 | 29/39 | 39/39 | 10/39 | |
10/2018 - 06/2022 | 25/42 | 32/42 | 42/42 | 10/42 | |
Sentinel-3A+B: | |||||
10/2018 - 06/2022 | 36/42 | 32/42 | 42/42 | 20/42 |
Percentiles were calculated based on the comparison of the TCDR using the Advanced Along Track Scanning Radiometer (AATSR) instrument against CERES as reference dataset for the variables Surface Incoming Shortwave Radiation (SIS), Surface Reflected Shortwave Radiation (SRS), Surface Outgoing Longwave Radiation (SOL) and Surface Downwelling Longwave Radiation (SDL). Percentiles were based on the time from 2002-2012 with monthly means and applied to the ICDR from 01/2017 (10/2018) to 06/2022 for Sentinel-3A (Sentinel-3B and merged product Sentinel-3A+B) based on measurements of the Sea and Land Surface Temperature Radiometer (SLSTR).
Most of the ICDR months are outside the TCDR-based KPI limits and leading to “bad” KPI tests. Therefore, the ICDR is not stable in relation to the TCDR. This is due to multiple reasons starting with the fact of a five year gap (2012-2016) between TCDR and ICDR. In addition, TCDR and ICDR are based on different instruments with SLSTR on Sentinel-3 and (A)ATSR/ATSR-2 on Envisat/ERS-2, respectively. Differences occur due to a lower bias between ICDR and reference dataset and a subtraction of the monthly means (based on the TCDR) to remove the annual cycle leads to values outside of the KPI range (see method in [D5], chapter 3.2.2). Please note that significant changes between 01/2017 - 12/2020 and 01/2017 - 12/2021 are due to bugfixes.
4. Compliance with user requirements
There are no direct user requirements for the Surface Radiation Budget defined in the Cloud_cci project. Looking at the GCOS ECV requirements for Surface Radiation Budget2 the values for SIS and SDL are 1 W/m2 uncertainty, while the TCDR dataset achieves an accuracy of 8 W/m2 for SIS and 12 W/m2 for SDL therefore they currently do not meet the GCOS requirements. Please find more detailed information about the target requirements in the corresponding (Target Requirement and Gap Analysis Document) TRGAD [D3].
ICDR accuracies (estimated with the first 5 years (2017 -2021)) appear to be better than TCDR accuracies. ICDR accuracies have been estimated in comparison with CERES surface fluxes. The better agreement of the ICDR dataset with CERES could come from different factors: (i) comparison of satellite vs satellite instead of satellite vs ground measurements; (ii) wider swath of SLSTR measurements; (iii) closest assumption (auxiliary data) used by both CERES and C3S estimate in the period considered (SRS and SOL strongly depend on surface reflectance and emissivity).
Table 4-1 provides an overview of the GCOS requirements for the surface radiative balance and the values achieved by the TCDR parameters. ICDR accuracies, estimated with 5 years of comparison with a similar satellite dataset, show results consistent with TCDR. It should be noted that GCOS requirements are targets and are often not attainable using existing or historical observing systems. The Cloud_cci doesn’t meet the requirement for resolving the diurnal cycle due to the nature of the satellite observations, but exceeds the spatial resolution.
Table 4-1: GCOS targets for Earth Radiation Budget ECVs and CDR values. TCDR values taken from Table 5-4 and Table 5-5 in [D1]. Anchor table4_1 table4_1
Product name |
| GCOS targets | Cloud_cci dataset |
SIS | Frequency | Monthly (resolving diurnal cycles) | Cloud_cci products do not meet the requirement for resolving the diurnal cycle. |
Resolution | 100 km | Cloud_cci products exceed the spatial resolution. | |
Measurement uncertainty | 1 W/m² on global mean | Uncertainty: 8.2 W/m² Standard Deviation: 24 W/m² on global mean (Validation with BSRN ground base measurements) | |
Stability | 0.2 W/m²/decade | 0.97 W/m²/decade (Comparison with CERES) | |
SDL | Frequency | Monthly (resolving diurnal cycles) | Cloud_cci products do not meet the requirement for resolving the diurnal cycle. |
Resolution | 100 km | Cloud_cci products exceed the spatial resolution. | |
Measurement uncertainty | 1 W/m² on global mean | Uncertainty: 12 W/m2 Standard Deviation: 15 W/m2 on global mean (Validation with BSRN ground base measurements) | |
Stability | 0.2 W/m²/decade | 2.76 W/m2/decade (Comparison with CERES) |
Known limitations [From D1 table 7.1]:
- Higher uncertainties in twilight conditions, especially in the shortwave fluxes, due to limitation in retrieving cloud optical thickness and cloud particle effective radius (input to the radiation calculation) in these conditions
- Partly sparse temporal/spatial sampling (partly compensated by introduced diurnal cycles correction).
- Downwelling longwave fluxes seem biased high.
Info | ||
---|---|---|
| ||
2 https://gcos.wmo.int/en/essential-climate-variables/surface-radiation/ecv-requirements(Last accessed on 28/02/2023) |
Anchor | ||||
---|---|---|---|---|
|
n/a
...
Product name
...
TCDR Accuracy [W/m2]
...
ICDR Accuracy [W/m2]
...
Surface Incoming Shortwave radiation (SIS)
...
8.2
...
2.3
...
Surface Reflected Shortwave radiation (SRS)
...
4.6
...
1.4
...
Surface Net Shortwave radiation (SNS)
...
13
...
3.7
...
Surface Outgoing Longwave radiation (SOL)
...
11
...
0.8
...
Surface Downwelling Longwave radiation (SDL)
...
12
...
8.6
...
Surface Net Longwave radiation (SNL)
...
23
...
9.4
...
Surface Radiation Budget (SRB)
...
36
...
13
1. Product validation methodology
The validation methodology is described in section 2.4 of [D1]. In summary, we use the bias (mean difference) between CDR and reference data as the metric for the accuracy. The bias corrected root mean squared error (bc-RMSE) is used to express the precision of CDR compared to a reference data record, which is also known as the standard deviation about the mean. Ground data from Baseline Surface Radiation Network (BSRN) stations are considered as a validation reference, and satellite estimates (e.g. the CERES surface radiation dataset) are considered for the comparison. Stability is calculated as the variation of the bias over a multi-annual time period.
2. Validation results
The TCDR validation results with BSRN and the comparison with CERES are provided in [D1], section 3.3.2, 5.3 and 5.4. Here below we present a summary of it. SIS and SDL are validated with BSRN (and these are the accuracies reported in Table 1) and compared with CERES. The evaluation with CERES is considered to be a comparison because CERES surface radiation dataset has a similar accuracy as the CDR dataset. Note that SOL TCDR data accuracy is only estimated in comparison with CERES (and this is the accuracy reported in Table 1).
The ICDR data of SIS, SRS, SOL and SDL are compared with CERES, using same methodology in [D1] section 5.3 and 5.4, and results are presented here in section 2.2.
There is no direct validation (e.g. against more accurate measurements) for the net fluxes and these accuracies are estimated by error propagation. Sections 2.4, 2.5, 2.6 present the estimate of the accuracy for SNS, SNL and SRB. Table 1 in the Executive summary provides a summary of the CDR accuracies. Table 2 summarizes the methodology used to estimate the accuracies for each product.
...
Product name
...
Validation with BSRN
...
Comparison with CERES
...
Uncertainty propagation
...
Surface Incoming Shortwave radiation (SIS)
...
TCDR
...
TCDR and ICDR
...
Surface Reflected Shortwave radiation (SRS)
...
TCDR and ICDR
...
TCDR and ICDR
...
Surface Net Shortwave radiation (SNS)
...
TCDR and ICDR
...
Surface Outgoing Longwave radiation (SOL)
...
TCDR and ICDR
...
Surface Downwelling Longwave radiation (SDL)
...
TCDR
...
TCDR and ICDR
...
Surface Net Longwave radiation (SNL)
...
TCDR and ICDR
...
Surface Radiation Budget (SRB)
...
TCDR and ICDR
2.1 Validation with BSRN ground base radiative flux
BSRN stations measure direct, diffuse and global downwelling shortwave and longwave fluxes with a 1 minute temporal resolution. The 1-min data was aggregated to monthly averages which were used for the validation. Using the CDR and the reference datasets (for multiple locations around the world) we compute the bias and standard deviation.
The validation for Surface Incoming Shortwave radiation (SIS) and Surface Downwelling Longwave radiation (SDL) with BSRN ground measurements is described in section 3.3.2 of [D1].
Validation of BOA fluxes against BSRN stations result in a standard deviation of 24 W/m² and a bias of 8.2 W/m² for shortwave radiation (SIS) and a standard deviation of 14 W/m² and a bias of 11.9 W/m² for longwave radiation (SDL).
Figure 1 and 2 show the results of the TCDR comparison with the BSRN incoming shortwave (SIS) and longwave (SDL) radiation with scatter plots and global maps showing the bias for each station.
...
Figure 1: Results from [D1]; Top: Validation results for Cloud_cci surface incoming shortwave flux using BSRN as a reference. Bottom: Bias for each ground station.
...
Figure 2: Results from [D1]; Top: Validation results for Cloud_cci surface downwelling longwave flux using BSRN as a reference. Bottom: Bias for each ground station.
2.2 Comparison with CERES satellite data
The TCDR and reference datasets are compared by means of multi-annual mean and standard deviation, all for a common time period (2003-2011). Global maps of multiannual Surface Incoming Shortwave radiation (SIS) and Surface Downwelling Longwave radiation (SDL) are computed for the TCDR and reference dataset. The scores (bias and bc-RMSE) are calculated by including all valid data points (for a latitude band of 60°S - 60°N) pairwise in the CERES and the Cloud_cci dataset.
The validation for Surface Incoming Shortwave radiation (SIS) and Surface Downwelling Longwave radiation (SDL) with CERES is described in section 5.3 and 5.4 of [D1]. Intercomparison of Cloud_cci radiation products with CERES present bias of 1.53 W/m² for SIS and bias of 10.17 W/m2 for SDL.
The stability estimated for the TCDR dataset are 0.97 and 2.76 W/m²/decade for SIS and SDL. The dataset is relatively stable during this period but shows some variations in the downwelling longwave radiation.
The same methodology is used to estimate the accuracy of SOL in comparison with CERES. Intercomparison of Cloud_cci radiation products (TCDR) with CERES present bias of 11 W/m² for SOL, which is the accuracy value reported in Table 1.
The same methodology is used to estimate the SIS, SRS, SOL and SDL accuracy of the ICDR.
Figures 3 and 4 show an example of ICDR products for March 2017 and the equivalent monthly mean product from CERES.
...
Figure 3: SRS, SOL, SIS and SDL values from SLSTR (ICDR dataset) for March 2017.
...
Figure 4. SRS, SOL, SIS and SDL values from CERES dataset for March 2017.
Bias between TCDR and CERES are calculated including all valid data points (for a latitude band of 60° S-60° N). Intercomparison of ICDR radiation products with CERES has been performed using the data between January 2017 and December 2018. The resulting bias (reported in Table 1) are: 2.3 W/m² for SIS, -1.4 W/m2 for SRS, 0.8 W/m2 for SOL and 8.6 W/m2 for SDL. Note that these results for ICDR are referring to only 24 months of the dataset and updated values will be computed in following reports.
General findings:
SIS (from [D1] section 5.3)
- The CDR dataset shows very similar patterns to the other Cloud_cci datasets of the global mean bottom of the atmosphere incoming solar radiation. Larger values are found for the subtropics; the maximum is located in the Atacama Desert. Lowest mean BOA incoming solar radiation is found over the polar regions.
- Spread among all the Cloud_cci datasets is highest for polar regions and high latitudes. Also the stratocumulus regions are clearly noticeable in the dataset.
- CDR dataset compared to CERES present similar patterns of temporal variability. The tropics show the lowest variability over time, while polar landmasses show the highest. CERES EBAF-SURFACE Ed4.0 contains both the highest and lowest values for temporal variability, but CDR and the other Cloud_cci datasets are alike.
- For the period from 2003 to 2011 no significant trends or anomalies are determinable. CDR dataset is relatively stable during this period and only show small variations in the BOA incoming solar radiation.
SDL (from [D1] section 5.4):
- Multi-annual global means of CDR and all Cloud_cci datasets compare very well with each other and hardly any larger differences are visible. The highest BOA downwelling thermal radiation is found over the tropics and subtropics, lowest values are found in Antarctica.
- With the exception of the inner tropics, there are slight differences between sea and land BOA downwelling thermal radiation in all datasets. Higher values are measured over the ocean than over land. Over land, mountains such as the Himalayas or the Andes are noticeable due to their clearly lower values.
- Similar to the global mean, the temporal variability of CDR datasets also shows a good agreement. In addition, the variability over land is significantly higher than over the ocean, with the exception of a narrow band in the inner tropics. The highest variability is found in East Asia.
- Strong seasonal cycles are visible with higher values in boreal summer and lower values in boreal winter. CDR, as well as all Cloud_cci datasets, time series also contain a small positive trend.
2.3 Surface Reflected Shortwave Radiation (SRS)
The TCDR accuracy of SRS is estimated from the accuracy of the Surface Incoming Shortwave radiation (SIS) and the Surface albedo (SAL).
Applying the error propagation, the accuracy of the SRS product can be estimated as:
Mathdisplay |
---|
\Delta SRS= \frac{\delta SRS}{\delta SIS} \Delta SIS + \frac{\delta SRS}{\delta SAL} \Delta SAL = SAL \Delta SIS + SIS \Delta SAL, \quad (Eq. 1) |
Where
Mathinline |
---|
\Delta SIS |
comes from [D1] and DSAL is considered as 25% of the SAL value. SAL is estimated as the ratio between Surface Reflected Shortwave radiation (SRS) and Surface Incoming Shortwave radiation (SIS).
Mathdisplay |
---|
SAL = SRS / SIS, \quad (Eq. 2) |
The resulting global mean accuracy for the TCDR SRS is 4.6 W/m2.
Accuracy of ICDR SRS is -1.4 W/m2 and is estimated in comparison with CERES (section 2.2).
2.4 Surface Net Shortwave Radiation (SNS)
The Surface Net Shortwave radiation (SNS) is calculated using:
Mathdisplay |
---|
SNS = SIS - SRS, \quad (Eq. 3) |
And the accuracy will be estimated as:
Mathdisplay |
---|
\Delta SNS = \Delta SIS + \Delta SRS, \quad (Eq. 4) |
The resulting global mean accuracy for the SNS is 13 W/m2 and 3.7 W/m2 for TCDR and ICDR respectively.
2.5 Surface Net Longwave Radiation (SNL)
Surface Net Longwave radiation (SNL) is calculated [D2] from:
Mathdisplay |
---|
SNL = SDL - SOL, \quad (Eq. 5) |
The accuracy
Mathinline |
---|
\Delta SNL |
will be estimated as:
Mathdisplay |
---|
\Delta SNL = \Delta SDL + \Delta SOL, \quad (Eq. 6) |
The resulting global mean accuracy for the SNL is 23 W/m2 for TCDR and 9.4 W/m2 for ICDR.
2.6 Surface Radiation Budget (SRB)
The total Surface Radiation Budget (SRB) is the sum of the short and longwave contributions:
Mathdisplay |
---|
SRB = SNS + SNL, \quad (Eq. 7) |
The accuracy will be estimated as:
Mathdisplay |
---|
\Delta SRB = \Delta SNS + \Delta SNL, \quad (Eq. 8) |
The resulting global mean accuracy for the SRB is 36 W/m2 for TCDR and 13 W/m2 for ICDR.
3. Application(s) specific assessment
...
4. Compliance with user requirements
There are no direct user requirements for the Surface Radiation Budget defined in the Cloud_cci project. Looking at the GCOS ECV requirements for Surface Radiation Budget (https://gcos.wmo.int/en/essential-climate-variables/surface-radiation/ecv-requirements) the values for SIS and SDL are 1 W/m2 uncertainty, while the TCDR dataset achieves an accuracy of 8 W/m2 for SIS and 12 W/m2 for SDL.
ICDR accuracies (estimated with the first 24 months only) appear to be better than TCDR accuracies. ICDR accuracies have been estimated in comparison with CERES surface fluxes. The better agreement of the ICDR dataset with CERES could come from different factors: (i) comparison of satellite vs satellite instead of satellite vs ground measurements; (ii) wider swath of SLSTR measurements; (iii) closest assumption (auxiliary data) used by both CERES and C3S estimate in the period considered (SRS and SOL strongly depend on surface reflectance and emissivity)
Table 3 provides an overview of the GCOS requirements for the surface radiative balance and the values achieved by the TCDR parameters. ICDR accuracies, estimated with 24 months of comparison with similar satellite dataset, show consistent results with TCDR. It should be noted that GCOS requirements are targets and are often not attainable using existing or historical observing systems. The Cloud_cci doesn’t meet the requirement for resolving the diurnal cycle due to the nature of the satellite observations, but exceeds the spatial resolution.
Known limitations [From D1 table 7.1]:
- Higher uncertainties in twilight conditions, especially in the shortwave fluxes, due to limitation in retrieving cloud optical thickness and effective radius (input to the radiation calculation) in these condition
- Partly spare temporal/spatial sampling (partly compensated by introduced diurnal cycles correction).
- Downwelling longwave fluxes seem biased high.
...
Variable
...
GCOS Targets
...
Cloud_cci accuracy and stability
...
SIS
...
SDL
...
Loeb, N.G., Doelling, D.R., Wang, H., Su, W., Nguyen, C., Corbett, J.G., Liang, L., Mitrescu, C., Rose, F.G., and Kato, S.: Clouds and the Earth's Radiant Energy System (CERES) Energy Balanced and Filled (EBAF) Top-of-Atmosphere (TOA) Edition 4.0 Data Product, J.Climate, 31(2), 895–918, https://doi.org/10.1175/JCLI-D-17-0208.1.
Poulsen, C. A., McGarragh, G. R., Thomas, G. E., Stengel, M., Christensen, M. W., Povey, A. C., Proud, S. R., Carboni, E., Hollmann, R., and Grainger, R. G.: Cloud_cci ATSR-2 and AATSR data set version 3: a 17-year climatology of global cloud and radiation properties, Earth Syst. Sci. Data, 12, 2121–2135, 2020, https://doi.org/10.5194/essd-12-2121-2020.
Stengel, M., Stapelberg, S., Schlundt, C., Karlsson, K.-G., Meirink, J.F., Poulsen, C., Bojanowski, J., and Stöckli, R.: Cloud_cci Product validation and Intercomparison Report, http://www.esa-cloud-cci.org/sites/default/files/upload/Cloud_cci_D4.1_PVIR_v6.1.pdf.
Info | ||
---|---|---|
| ||
This document has been produced with funding by the European Union in the context of the Copernicus Climate Change Service (C3S) .The activities leading to these results have been contracted, operated by the European Centre for Medium-Range Weather Forecasts , operator of C3Son behalf ofon the European Union ( Delegation agreementContribution Agreement signed on 1122/ 1107/ 20142021). All information in this document is provided "as is" and no guarantee orof warranty is given that the information is fit for any particular purpose. The users thereof use the information at their sole risk and liability. For the avoidance of all doubt, the European Commission and the European Centre for Medium-Range Weather Forecasts have no liability in respect of this document, which is merely representing the author's view. |
Related articles
Content by Label | ||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|