Contributors: E. Carboni (UKRI-STFC RAL Space), G. Thomas (UKRI-STFC RAL Space)
Issued by: STFC RAL Space / Elisa Carboni
Date: 10/03/2021
Ref: C3S_D312b_Lot1.2.2.6-v3.2_202103_PQAR_CCISurfaceRadiationBudget_v1.0
Official reference number service contract: 2018/C3S_312b_Lot1_DWD/SC1
History of modifications
List of datasets covered by this document
Related documents
Acronyms
Scope of the document
This document provides a description of the product validation results for the Essential Climate Variable (ECV) Surface Radiation Budget. These products are brokered to (in case of (A)ATSR) or produced for the Climate Data Store (in the case of SLSTR) by the Copernicus Climate Change Service (C3S).
The TCDR is a brokered version of ESA's Cloud_cci ATSR2-AATSR version 3.0 (Level-3C) dataset, produced by STFC RAL Space from the second Along-Track Scanning Radiometer (ATSR-2) onboard the second European Remote Sensing Satellite (ERS-2) spanning 1995-2003 and the Advanced ATSR (AATSR) onboard ENVISAT, which spanned 2002-2012. The ICDR is produced by C3S, using SLSTR onboard of Sentinel-3 and spans January 2017 to present (with a validation period from January/2017 – December/2018 in this version of the document).
Executive Summary
The ESA Climate Change Initiative (CCI) Surface Radiation Budget Climate Data Record (CDR) is a brokered product from the ESA Cloud_cci project, while the extension Interim CDR (ICDR), produced from the Sea and Land Surface Temperature Radiometer (SLSTR), is produced specifically for C3S. The product is generated by STFC RAL Space, using the Community Cloud for Climate (CC4CL) processor, based on the Optimal Retrieval of Aerosol and Cloud (ORAC) algorithm. The Surface Radiation Budget is a product of the Broadband Radiative Flux Retrieval (BRFR) module of CC4CL, which uses the cloud properties produced by ORAC to compute broadband radiative flux values.
The Cloud_cci dataset comprises 17 years (1995-2012) of satellite-based measurements derived from the Along Track Scanning Radiometers (ATSR-2 and AATSR) onboard the ESA second European Research Satellite (ERS-2) and ENVISAT satellites. This TCDR is partnered with the ICDR produced from the Sentinel-3A SLSTR, beginning in 2017, and Sentinel-3B SLSTR beginning in October 2018. The dataset encompasses level-3 data (monthly means) on a regular global latitude-longitude grid (with a resolution of 0.5° × 0.5°) and includes these products: the Surface Incoming and Reflected Shortwave radiation (SIS and SRS respectively), the Surface Downwelling and Outgoing Longwave radiation (SDL and SOL respectively), the Surface Net Shortwave and Longwave radiation (SNS and SNL), and the total Surface Radiation Budget (SRB).
Table 1 provides a summary of the calculated accuracies.
Table:1 Summary of the accuracy of the Surface Radiation Budget dataset. The 'bold' accuracies come from direct validation with a ground measurement network, the other come from the intercomparison with similar datasets or with an uncertainty propagation, as explained in section 2. ICDR values are obtained from data between January 2017 and December 2018.
Product name | TCDR Accuracy [W/m2] | ICDR Accuracy [W/m2] |
Surface Incoming Shortwave radiation (SIS) | 8.2 | 2.3 |
Surface Reflected Shortwave radiation (SRS) | 4.6 | 1.4 |
Surface Net Shortwave radiation (SNS) | 13 | 3.7 |
Surface Outgoing Longwave radiation (SOL) | 11 | 0.8 |
Surface Downwelling Longwave radiation (SDL) | 12 | 8.6 |
Surface Net Longwave radiation (SNL) | 23 | 9.4 |
Surface Radiation Budget (SRB) | 36 | 13 |
1. Product validation methodology
The validation methodology is described in section 2.4 of [D1]. In summary, we use the bias (mean difference) between CDR and reference data as the metric for the accuracy. The bias corrected root mean squared error (bc-RMSE) is used to express the precision of CDR compared to a reference data record, which is also known as the standard deviation about the mean. Ground data from Baseline Surface Radiation Network (BSRN) stations are considered as a validation reference, and satellite estimates (e.g. the CERES surface radiation dataset) are considered for the comparison. Stability is calculated as the variation of the bias over a multi-annual time period.
2. Validation results
The TCDR validation results with BSRN and the comparison with CERES are provided in [D1], section 3.3.2, 5.3 and 5.4. Here below we present a summary of it. SIS and SDL are validated with BSRN (and these are the accuracies reported in Table 1) and compared with CERES. The evaluation with CERES is considered to be a comparison because CERES surface radiation dataset has a similar accuracy as the CDR dataset. Note that SOL TCDR data accuracy is only estimated in comparison with CERES (and this is the accuracy reported in Table 1).
The ICDR data of SIS, SRS, SOL and SDL are compared with CERES, using same methodology in [D1] section 5.3 and 5.4, and results are presented here in section 2.2.
There is no direct validation (e.g. against more accurate measurements) for the net fluxes and these accuracies are estimated by error propagation. Sections 2.4, 2.5, 2.6 present the estimate of the accuracy for SNS, SNL and SRB. Table 1 in the Executive summary provides a summary of the CDR accuracies. Table 2 summarizes the methodology used to estimate the accuracies for each product.
Table 2: Summary of methodologies used to estimate the accuracies, for TCDR and ICDR datasets.
Product name | Validation with BSRN | Comparison with CERES | Uncertainty propagation |
Surface Incoming Shortwave radiation (SIS) | TCDR | TCDR and ICDR | |
Surface Reflected Shortwave radiation (SRS) | TCDR and ICDR | TCDR and ICDR | |
Surface Net Shortwave radiation (SNS) | TCDR and ICDR | ||
Surface Outgoing Longwave radiation (SOL) | TCDR and ICDR | ||
Surface Downwelling Longwave radiation (SDL) | TCDR | TCDR and ICDR | |
Surface Net Longwave radiation (SNL) | TCDR and ICDR | ||
Surface Radiation Budget (SRB) | TCDR and ICDR |
2.1 Validation with BSRN ground base radiative flux
BSRN stations measure direct, diffuse and global downwelling shortwave and longwave fluxes with a 1 minute temporal resolution. The 1-min data was aggregated to monthly averages which were used for the validation. Using the CDR and the reference datasets (for multiple locations around the world) we compute the bias and standard deviation.
The validation for Surface Incoming Shortwave radiation (SIS) and Surface Downwelling Longwave radiation (SDL) with BSRN ground measurements is described in section 3.3.2 of [D1].
Validation of BOA fluxes against BSRN stations result in a standard deviation of 24 W/m² and a bias of 8.2 W/m² for shortwave radiation (SIS) and a standard deviation of 14 W/m² and a bias of 11.9 W/m² for longwave radiation (SDL).
Figure 1 and 2 show the results of the TCDR comparison with the BSRN incoming shortwave (SIS) and longwave (SDL) radiation with scatter plots and global maps showing the bias for each station.
Figure 1: Results from [D1]; Top: Validation results for Cloud_cci surface incoming shortwave flux using BSRN as a reference. Bottom: Bias for each ground station.
Figure 2: Results from [D1]; Top: Validation results for Cloud_cci surface downwelling longwave flux using BSRN as a reference. Bottom: Bias for each ground station.
2.2 Comparison with CERES satellite data
The TCDR and reference datasets are compared by means of multi-annual mean and standard deviation, all for a common time period (2003-2011). Global maps of multiannual Surface Incoming Shortwave radiation (SIS) and Surface Downwelling Longwave radiation (SDL) are computed for the TCDR and reference dataset. The scores (bias and bc-RMSE) are calculated by including all valid data points (for a latitude band of 60°S - 60°N) pairwise in the CERES and the Cloud_cci dataset.
The validation for Surface Incoming Shortwave radiation (SIS) and Surface Downwelling Longwave radiation (SDL) with CERES is described in section 5.3 and 5.4 of [D1]. Intercomparison of Cloud_cci radiation products with CERES present bias of 1.53 W/m² for SIS and bias of 10.17 W/m2 for SDL.
The stability estimated for the TCDR dataset are 0.97 and 2.76 W/m²/decade for SIS and SDL. The dataset is relatively stable during this period but shows some variations in the downwelling longwave radiation.
The same methodology is used to estimate the accuracy of SOL in comparison with CERES. Intercomparison of Cloud_cci radiation products (TCDR) with CERES present bias of 11 W/m² for SOL, which is the accuracy value reported in Table 1.
The same methodology is used to estimate the SIS, SRS, SOL and SDL accuracy of the ICDR.
Figures 3 and 4 show an example of ICDR products for March 2017 and the equivalent monthly mean product from CERES.
Figure 3: SRS, SOL, SIS and SDL values from SLSTR (ICDR dataset) for March 2017.
Figure 4. SRS, SOL, SIS and SDL values from CERES dataset for March 2017.
Bias between TCDR and CERES are calculated including all valid data points (for a latitude band of 60° S-60° N). Intercomparison of ICDR radiation products with CERES has been performed using the data between January 2017 and December 2018. The resulting bias (reported in Table 1) are: 2.3 W/m² for SIS, -1.4 W/m2 for SRS, 0.8 W/m2 for SOL and 8.6 W/m2 for SDL. Note that these results for ICDR are referring to only 24 months of the dataset and updated values will be computed in following reports.
General findings:
SIS (from [D1] section 5.3)
- The CDR dataset shows very similar patterns to the other Cloud_cci datasets of the global mean bottom of the atmosphere incoming solar radiation. Larger values are found for the subtropics; the maximum is located in the Atacama Desert. Lowest mean BOA incoming solar radiation is found over the polar regions.
- Spread among all the Cloud_cci datasets is highest for polar regions and high latitudes. Also the stratocumulus regions are clearly noticeable in the dataset.
- CDR dataset compared to CERES present similar patterns of temporal variability. The tropics show the lowest variability over time, while polar landmasses show the highest. CERES EBAF-SURFACE Ed4.0 contains both the highest and lowest values for temporal variability, but CDR and the other Cloud_cci datasets are alike.
- For the period from 2003 to 2011 no significant trends or anomalies are determinable. CDR dataset is relatively stable during this period and only show small variations in the BOA incoming solar radiation.
SDL (from [D1] section 5.4):
- Multi-annual global means of CDR and all Cloud_cci datasets compare very well with each other and hardly any larger differences are visible. The highest BOA downwelling thermal radiation is found over the tropics and subtropics, lowest values are found in Antarctica.
- With the exception of the inner tropics, there are slight differences between sea and land BOA downwelling thermal radiation in all datasets. Higher values are measured over the ocean than over land. Over land, mountains such as the Himalayas or the Andes are noticeable due to their clearly lower values.
- Similar to the global mean, the temporal variability of CDR datasets also shows a good agreement. In addition, the variability over land is significantly higher than over the ocean, with the exception of a narrow band in the inner tropics. The highest variability is found in East Asia.
- Strong seasonal cycles are visible with higher values in boreal summer and lower values in boreal winter. CDR, as well as all Cloud_cci datasets, time series also contain a small positive trend.
2.3 Surface Reflected Shortwave Radiation (SRS)
The TCDR accuracy of SRS is estimated from the accuracy of the Surface Incoming Shortwave radiation (SIS) and the Surface albedo (SAL).
Applying the error propagation, the accuracy of the SRS product can be estimated as:
\[ \Delta SRS= \frac{\delta SRS}{\delta SIS} \Delta SIS + \frac{\delta SRS}{\delta SAL} \Delta SAL = SAL \Delta SIS + SIS \Delta SAL, \quad (Eq. 1) \]Where \( \Delta SIS \) comes from [D1] and DSAL is considered as 25% of the SAL value. SAL is estimated as the ratio between Surface Reflected Shortwave radiation (SRS) and Surface Incoming Shortwave radiation (SIS).
\[ SAL = SRS / SIS, \quad (Eq. 2) \]The resulting global mean accuracy for the TCDR SRS is 4.6 W/m2.
Accuracy of ICDR SRS is -1.4 W/m2 and is estimated in comparison with CERES (section 2.2).
2.4 Surface Net Shortwave Radiation (SNS)
The Surface Net Shortwave radiation (SNS) is calculated using:
\[ SNS = SIS - SRS, \quad (Eq. 3) \]And the accuracy will be estimated as:
\[ \Delta SNS = \Delta SIS + \Delta SRS, \quad (Eq. 4) \]The resulting global mean accuracy for the SNS is 13 W/m2 and 3.7 W/m2 for TCDR and ICDR respectively.
2.5 Surface Net Longwave Radiation (SNL)
Surface Net Longwave radiation (SNL) is calculated [D2] from:
\[ SNL = SDL - SOL, \quad (Eq. 5) \]The accuracy \( \Delta SNL \) will be estimated as:
\[ \Delta SNL = \Delta SDL + \Delta SOL, \quad (Eq. 6) \]The resulting global mean accuracy for the SNL is 23 W/m2 for TCDR and 9.4 W/m2 for ICDR.
2.6 Surface Radiation Budget (SRB)
The total Surface Radiation Budget (SRB) is the sum of the short and longwave contributions:
\[ SRB = SNS + SNL, \quad (Eq. 7) \]The accuracy will be estimated as:
\[ \Delta SRB = \Delta SNS + \Delta SNL, \quad (Eq. 8) \]The resulting global mean accuracy for the SRB is 36 W/m2 for TCDR and 13 W/m2 for ICDR.
3. Application(s) specific assessment
N/A
4. Compliance with user requirements
There are no direct user requirements for the Surface Radiation Budget defined in the Cloud_cci project. Looking at the GCOS ECV requirements for Surface Radiation Budget (https://gcos.wmo.int/en/essential-climate-variables/surface-radiation/ecv-requirements) the values for SIS and SDL are 1 W/m2 uncertainty, while the TCDR dataset achieves an accuracy of 8 W/m2 for SIS and 12 W/m2 for SDL.
ICDR accuracies (estimated with the first 24 months only) appear to be better than TCDR accuracies. ICDR accuracies have been estimated in comparison with CERES surface fluxes. The better agreement of the ICDR dataset with CERES could come from different factors: (i) comparison of satellite vs satellite instead of satellite vs ground measurements; (ii) wider swath of SLSTR measurements; (iii) closest assumption (auxiliary data) used by both CERES and C3S estimate in the period considered (SRS and SOL strongly depend on surface reflectance and emissivity)
Table 3 provides an overview of the GCOS requirements for the surface radiative balance and the values achieved by the TCDR parameters. ICDR accuracies, estimated with 24 months of comparison with similar satellite dataset, show consistent results with TCDR. It should be noted that GCOS requirements are targets and are often not attainable using existing or historical observing systems. The Cloud_cci doesn’t meet the requirement for resolving the diurnal cycle due to the nature of the satellite observations, but exceeds the spatial resolution.
Known limitations [From D1 table 7.1]:
- Higher uncertainties in twilight conditions, especially in the shortwave fluxes, due to limitation in retrieving cloud optical thickness and effective radius (input to the radiation calculation) in these condition
- Partly spare temporal/spatial sampling (partly compensated by introduced diurnal cycles correction).
- Downwelling longwave fluxes seem biased high.
Table 3: GCOS targets for Surface Radiation Budget ECVs and TCDR values. TCDR values taken from Table 5-4 and Table 5-5 in [D1].
Variable | GCOS Targets | Cloud_cci accuracy and stability |
SIS |
|
|
SDL |
|
|
References
Loeb, N.G., Doelling, D.R., Wang, H., Su, W., Nguyen, C., Corbett, J.G., Liang, L., Mitrescu, C., Rose, F.G., and Kato, S.: Clouds and the Earth's Radiant Energy System (CERES) Energy Balanced and Filled (EBAF) Top-of-Atmosphere (TOA) Edition 4.0 Data Product, J.Climate, 31(2), 895–918, https://doi.org/10.1175/JCLI-D-17-0208.1.
Poulsen, C. A., McGarragh, G. R., Thomas, G. E., Stengel, M., Christensen, M. W., Povey, A. C., Proud, S. R., Carboni, E., Hollmann, R., and Grainger, R. G.: Cloud_cci ATSR-2 and AATSR data set version 3: a 17-year climatology of global cloud and radiation properties, Earth Syst. Sci. Data, 12, 2121–2135, 2020, https://doi.org/10.5194/essd-12-2121-2020.
Stengel, M., Stapelberg, S., Schlundt, C., Karlsson, K.-G., Meirink, J.F., Poulsen, C., Bojanowski, J., and Stöckli, R.: Cloud_cci Product validation and Intercomparison Report, http://www.esa-cloud-cci.org/sites/default/files/upload/Cloud_cci_D4.1_PVIR_v6.1.pdf.
