The Product Validation and Intercomparison Report [D1] includes the validation and intercomparison of the TCDR Surface Radiation Budget versus the CERES satellite dataset. The same methodology is used for the ICDR.

2. Validation results

The validation results for the TCDR products are presented and described in detail in [D1], sections 3.3.2, 5.3 and 5.4. In this document, a summary highlighting the main results is presented.  Only SIS and SDL in the TCDR product are validated with BSRN.   All other properties (including SIS and SDL in the ICDR) are compared with CERES. The evaluation with CERES is considered to be a comparison because the CERES surface radiation dataset has a similar accuracy to the CDR dataset.

 The ICDR data of SIS, SRS, SOL and SDL are compared with CERES, using the methodology described in [D1] section 5.3 and 5.4, and results are presented here in section 2.2.

There is no direct validation (e.g. against more accurate measurements) for the net fluxes and these accuracies are estimated by error propagation. Sections 2.4, 2.5, 2.6 in this document present the estimate of the accuracy for SNS, SNL and SRB. Table 2-1 provides a summary of the CDR accuracies.

2.1 Validation with BSRN ground base radiative flux

BSRN stations measure direct, diffuse and global downwelling shortwave and longwave fluxes with a 1-minute temporal resolution. The 1-minute data was aggregated to monthly averages which were used for the validation. Using the TCDR and the reference datasets (for multiple locations around the world (see Figures 2-1 and 2-2)) we compute the bias and standard deviation.

The validation for Surface Incoming Shortwave radiation (SIS) and Surface Downwelling Longwave radiation (SDL) with BSRN ground measurements is described in section 3.3.2 of [D1].

Validation of BOA fluxes (i.e. SIS and SDL) in the TCDR against BSRN stations result in a standard deviation of 24 W/m² and a bias of 8.2 W/m² for shortwave radiation (SIS) and a standard deviation of 14 W/m² and a bias of 11.9 W/m² for longwave radiation (SDL).

Figures 2-1 and 2-2 show the results of the TCDR comparison with the BSRN incoming shortwave (SIS) and longwave (SDL) radiation with scatter plots and global maps showing the bias for each station.

Image Added

Image Added

2.2 Comparison with CERES satellite data

The TCDR and reference datasets from CERES are compared by means of multi-annual mean and standard deviation, all for a common time period (2003-2011). Global maps of multiannual Surface Incoming Shortwave radiation (SIS) and Surface Downwelling Longwave radiation (SDL) are computed for the TCDR and reference dataset. The scores (bias and bc-RMSE) are calculated by including all valid data points (for a latitude band of 60°S - 60°N) pairwise in the CERES and the Cloud_cci dataset.

The validation for Surface Incoming Shortwave radiation (SIS) and Surface Downwelling Longwave radiation (SDL) with CERES is described in section 5.3 and 5.4 of [D1]. Intercomparison of Cloud_cci radiation products with CERES present bias of 1.53 W/m² for SIS and bias of 10.17 W/m² for SDL.

The stability estimated for the TCDR dataset are 0.97 and 2.76 W/m²/decade for SIS and SDL respectively. The dataset is relatively stable during this period but shows some variations in the downwelling longwave radiation.

The same methodology is used to estimate the accuracy of SOL in comparison with CERES. Intercomparison of Cloud_cci radiation products (TCDR) with CERES present bias of 11 W/m² for SOL, which is the accuracy value reported in Table 2-1.

The evaluation method is also used to estimate the SIS, SRS, SOL and SDL accuracy of the ICDR.

Figures 2-3 and 2-4 show an example of ICDR products for March 2017 and the equivalent monthly mean product from CERES.

Image Added

Image Added

Bias between TCDR and CERES are calculated including all valid data points (for a latitude band of 60° S-60° N). Intercomparison of ICDR radiation products with CERES has been performed using the data between January 2017 and March 2022. The resulting bias (reported in Table 2-1) are the following:

SLSTR-A: 1.8 W/m² for SIS, 1.6 W/m² for SRS, 1.6 W/m² for SOL and 9.7 W/m² for SDL

SLSTR-B: 0.23 W/m² for SIS, 2.1 W/m² for SRS, 4.1 W/m² for SOL, 11 W/m² for SDL 

SLSTR-A+B:  0.51 W/m² for SIS, 2.2 W/m² for SRS, 3.8 W/m² for SOL, 11 W/m² for SDL 

General findings:

SIS (from [D1] section 5.3)

• The CDR dataset shows very similar patterns to the other Cloud_cci datasets of the global mean bottom of the atmosphere incoming solar radiation. Larger values are found for the subtropics; the maximum is located in the Atacama Desert. Lowest mean BOA incoming solar radiation is found over the polar regions.
• Spread among all the Cloud_cci datasets is highest for polar regions and high latitudes. Also the stratocumulus regions are clearly noticeable in the dataset.
• CDR dataset compared to CERES present similar patterns of temporal variability. The tropics show the lowest variability over time, while polar landmasses show the highest. CERES EBAF-SURFACE Ed4.0 contains both the highest and lowest values for temporal variability, but CDR and the other Cloud_cci datasets are alike.
• For the period from 2003 to 2011 no significant trends or anomalies are determinable. CDR dataset is relatively stable during this period and only show small variations in the BOA incoming solar radiation.

SDL (from [D1] section 5.4):

• Multi-annual global means of CDR and all Cloud_cci datasets compare very well with each other and hardly any larger differences are visible. The highest BOA downwelling thermal radiation is found over the tropics and subtropics, lowest values are found in Antarctica.
• With the exception of the inner tropics, there are slight differences between sea and land BOA downwelling thermal radiation in all datasets. Higher values are measured over the ocean than over land. Over land, mountains such as the Himalayas or the Andes are noticeable due to their clearly lower values.
• Similar to the global mean, the temporal variability of CDR datasets also shows a good agreement. In addition, the variability over land is significantly higher than over the ocean, with the exception of a narrow band in the inner tropics. The highest variability is found in East Asia.
• Strong seasonal cycles are visible with higher values in boreal summer and lower values in boreal winter. CDR time series, as well as all Cloud_cci datasets, also contain a small positive trend.

2.3 Surface Reflected Shortwave Radiation (SRS)

The TCDR accuracy of SRS is estimated from the accuracy of the Surface Incoming Shortwave radiation (SIS) and the Surface albedo (SAL).

Applying the error propagation, the accuracy of the SRS product can be estimated as:

\Delta SRS= \frac{\delta SRS}{\delta SIS} \Delta SIS + \frac{\delta SRS}{\delta SAL} \Delta SAL = SAL \Delta SIS + SIS \Delta SAL, \quad \ \ (Eq. 3)

Where

\Delta SIS

comes from [D1] and DSAL is considered as 25% of the SAL value. SAL is estimated as the ratio between Surface Reflected Shortwave radiation (SRS) and Surface Incoming Shortwave radiation (SIS).

SAL = SRS / SIS, \quad \ \ (Eq. 4)

The resulting global mean accuracy for the TCDR SRS is 4.6 W/m².

Accuracies of ICDR SRS are 1.6, 2.1 and 2.2 W/m² for SLSTR-A, SLSTR-B and SLSTR A+B respectively and are estimated in comparison with CERES (section 2.2).

2.4 Surface Net Shortwave Radiation (SNS)

The Surface Net Shortwave radiation (SNS) is calculated using:

SNS = SIS - SRS, \quad \ \ (Eq. 5)

And the accuracy will be estimated as:

\Delta SNS = \Delta SIS + \Delta SRS, \quad \ \ (Eq. 6)

The resulting global mean accuracy for the SNS is 13 W/m² for TCDR, 3.4, 2.3 and 2.7 W/m² for ICDR SLSTR-A, SLSTR-B and SLSTR-A+B respectively.

2.5 Surface Net Longwave Radiation (SNL)

Surface Net Longwave radiation (SNL) is calculated [D2] from:

SNL = SDL - SOL, \quad \ \ (Eq. 7)

The accuracy

\Delta SNL

 will be estimated as:

\Delta SNL = \Delta SDL + \Delta SOL, \quad \ \ (Eq. 8)

The resulting global mean accuracy for the SNL is 23 W/m² for TCDR; 11 W/m² , 15 W/m² and  15 W/m² for ICDR SLSTR-A , B and A+B respectively.

2.6 Surface Radiation Budget (SRB)

The total Surface Radiation Budget (SRB) is the sum of the short and longwave contributions:

SRB = SNS + SNL, \quad \ \ (Eq. 9)

The accuracy will be estimated as:

\Delta SRB = \Delta SNS + \Delta SNL, \quad \ \ (Eq. 10)

The resulting global mean accuracy for the SRB is 36 W/m² for TCDR, 14 W/m² , 17 W/m² and 18 W/m² for ICDR SLSTR-A , B and A+B respectively.

3. Application(s) specific assessment

This section is not applicable. There are no additional application specific assessments known since the dataset has just been published.

4. Compliance with user requirements

There are no direct user requirements for the Surface Radiation Budget defined in the Cloud_cci project. Looking at the GCOS ECV requirements for Surface Radiation Budget² the values for SIS and SDL are 1 W/m² uncertainty, while the TCDR dataset achieves an accuracy of 8 W/m² for SIS and 12 W/m² for SDL therefore they currently do not meet the GCOS requirements. Please find more detailed information about the target requirements in the corresponding (Target Requirement and Gap Analysis Document) TRGAD [D3].

ICDR accuracies (estimated with the first 5 years (2017 -2021)) appear to be better than TCDR accuracies. ICDR accuracies have been estimated in comparison with CERES surface fluxes.  The better agreement of the ICDR dataset with CERES could come from different factors: (i) comparison of satellite vs satellite instead of satellite vs ground measurements; (ii) wider swath of SLSTR measurements; (iii) closest assumption (auxiliary data) used by both CERES and C3S estimate in the period considered (SRS and SOL strongly depend on surface reflectance and emissivity).  

Table 4-1 provides an overview of the GCOS requirements for the surface radiative balance and the values achieved by the TCDR parameters. ICDR accuracies, estimated with 5 years of comparison with a similar satellite dataset, show results consistent with TCDR. It should be noted that GCOS requirements are targets and are often not attainable using existing or historical observing systems. The Cloud_cci doesn’t meet the requirement for resolving the diurnal cycle due to the nature of the satellite observations, but exceeds the spatial resolution.

 Known limitations [From D1 table 7.1]:

• Higher uncertainties in twilight conditions, especially in the shortwave fluxes, due to limitation in retrieving cloud optical thickness and cloud particle effective radius (input to the radiation calculation) in these conditions
• Partly sparse temporal/spatial sampling (partly compensated by introduced diurnal cycles correction).
• Downwelling longwave fluxes seem biased high.

Anchor
reference reference

References

n/a

...

Product name

...

TCDR Accuracy [W/m²]

...

ICDR Accuracy [W/m²]

...

Surface Incoming Shortwave radiation (SIS)

...

8.2

...

2.3

...

Surface Reflected Shortwave radiation (SRS)

...

4.6

...

1.4

...

Surface Net Shortwave radiation (SNS)

...

13

...

3.7

...

Surface Outgoing Longwave radiation (SOL)

...

11

...

0.8

...

Surface Downwelling Longwave radiation (SDL)

...

12

...

8.6

...

Surface Net Longwave radiation (SNL)

...

23

...

9.4

...

Surface Radiation Budget (SRB)

...

36

...

13

1. Product validation methodology

The validation methodology is described in section 2.4 of [D1]. In summary, we use the bias (mean difference) between CDR and reference data as the metric for the accuracy. The bias corrected root mean squared error (bc-RMSE) is used to express the precision of CDR compared to a reference data record, which is also known as the standard deviation about the mean. Ground data from Baseline Surface Radiation Network (BSRN) stations are considered as a validation reference, and satellite estimates (e.g. the CERES surface radiation dataset) are considered for the comparison. Stability is calculated as the variation of the bias over a multi-annual time period.

2. Validation results

The TCDR validation results with BSRN and the comparison with CERES are provided in [D1], section 3.3.2, 5.3 and 5.4. Here below we present a summary of it. SIS and SDL are validated with BSRN (and these are the accuracies reported in Table 1) and compared with CERES. The evaluation with CERES is considered to be a comparison because CERES surface radiation dataset has a similar accuracy as the CDR dataset. Note that SOL TCDR data accuracy is only estimated in comparison with CERES (and this is the accuracy reported in Table 1).

The ICDR data of SIS, SRS, SOL and SDL are compared with CERES, using same methodology in [D1] section 5.3 and 5.4, and results are presented here in section 2.2.

There is no direct validation (e.g. against more accurate measurements) for the net fluxes and these accuracies are estimated by error propagation. Sections 2.4, 2.5, 2.6 present the estimate of the accuracy for SNS, SNL and SRB. Table 1 in the Executive summary provides a summary of the CDR accuracies. Table 2 summarizes the methodology used to estimate the accuracies for each product.

...

Product name

...

Validation with BSRN

...

Comparison with CERES

...

Uncertainty propagation

...

Surface Incoming Shortwave radiation (SIS)

...

TCDR

...

TCDR and ICDR

...

Surface Reflected Shortwave radiation (SRS)

...

TCDR and ICDR

...

TCDR and ICDR

...

Surface Net Shortwave radiation (SNS)

...

TCDR and ICDR

...

Surface Outgoing Longwave radiation (SOL)

...

TCDR and ICDR

...

Surface Downwelling Longwave radiation (SDL)

...

TCDR

...

TCDR and ICDR

...

Surface Net Longwave radiation (SNL)

...

TCDR and ICDR

...

Surface Radiation Budget (SRB)

...

TCDR and ICDR

2.1 Validation with BSRN ground base radiative flux

BSRN stations measure direct, diffuse and global downwelling shortwave and longwave fluxes with a 1 minute temporal resolution. The 1-min data was aggregated to monthly averages which were used for the validation. Using the CDR and the reference datasets (for multiple locations around the world) we compute the bias and standard deviation.

The validation for Surface Incoming Shortwave radiation (SIS) and Surface Downwelling Longwave radiation (SDL) with BSRN ground measurements is described in section 3.3.2 of [D1].

Validation of BOA fluxes against BSRN stations result in a standard deviation of 24 W/m² and a bias of 8.2 W/m² for shortwave radiation (SIS) and a standard deviation of 14 W/m² and a bias of 11.9 W/m² for longwave radiation (SDL).

Figure 1 and 2 show the results of the TCDR comparison with the BSRN incoming shortwave (SIS) and longwave (SDL) radiation with scatter plots and global maps showing the bias for each station.

...

Image Removed

Figure 1:  Results from [D1]; Top: Validation results for Cloud_cci surface incoming shortwave flux using BSRN as a reference. Bottom: Bias for each ground station.

...

Image Removed

Figure 2: Results from [D1]; Top: Validation results for Cloud_cci surface downwelling longwave flux using BSRN as a reference. Bottom: Bias for each ground station.

2.2 Comparison with CERES satellite data

The TCDR and reference datasets are compared by means of multi-annual mean and standard deviation, all for a common time period (2003-2011). Global maps of multiannual Surface Incoming Shortwave radiation (SIS) and Surface Downwelling Longwave radiation (SDL) are computed for the TCDR and reference dataset. The scores (bias and bc-RMSE) are calculated by including all valid data points (for a latitude band of 60°S - 60°N) pairwise in the CERES and the Cloud_cci dataset.

The validation for Surface Incoming Shortwave radiation (SIS) and Surface Downwelling Longwave radiation (SDL) with CERES is described in section 5.3 and 5.4 of [D1]. Intercomparison of Cloud_cci radiation products with CERES present bias of 1.53 W/m² for SIS and bias of 10.17 W/m² for SDL.

The stability estimated for the TCDR dataset are 0.97 and 2.76 W/m²/decade for SIS and SDL. The dataset is relatively stable during this period but shows some variations in the downwelling longwave radiation.

The same methodology is used to estimate the accuracy of SOL in comparison with CERES. Intercomparison of Cloud_cci radiation products (TCDR) with CERES present bias of 11 W/m² for SOL, which is the accuracy value reported in Table 1.

The same methodology is used to estimate the SIS, SRS, SOL and SDL accuracy of the ICDR.

Figures 3 and 4 show an example of ICDR products for March 2017 and the equivalent monthly mean product from CERES.

...

Image RemovedFigure 3: SRS, SOL, SIS and SDL values from SLSTR (ICDR dataset) for March 2017.

...

Image Removed

Figure 4. SRS, SOL, SIS and SDL values from CERES dataset for March 2017.

Bias between TCDR and CERES are calculated including all valid data points (for a latitude band of 60° S-60° N). Intercomparison of ICDR radiation products with CERES has been performed using the data between January 2017 and December 2018. The resulting bias (reported in Table 1) are: 2.3 W/m² for SIS, -1.4 W/m² for SRS, 0.8 W/m² for SOL and 8.6 W/m² for SDL. Note that these results for ICDR are referring to only 24 months of the dataset and updated values will be computed in following reports.

General findings:

 SIS (from [D1] section 5.3)

• The CDR dataset shows very similar patterns to the other Cloud_cci datasets of the global mean bottom of the atmosphere incoming solar radiation. Larger values are found for the subtropics; the maximum is located in the Atacama Desert. Lowest mean BOA incoming solar radiation is found over the polar regions.
• Spread among all the Cloud_cci datasets is highest for polar regions and high latitudes. Also the stratocumulus regions are clearly noticeable in the dataset.
• CDR dataset compared to CERES present similar patterns of temporal variability. The tropics show the lowest variability over time, while polar landmasses show the highest. CERES EBAF-SURFACE Ed4.0 contains both the highest and lowest values for temporal variability, but CDR and the other Cloud_cci datasets are alike.
• For the period from 2003 to 2011 no significant trends or anomalies are determinable. CDR dataset is relatively stable during this period and only show small variations in the BOA incoming solar radiation.

SDL (from [D1] section 5.4):

• Multi-annual global means of CDR and all Cloud_cci datasets compare very well with each other and hardly any larger differences are visible. The highest BOA downwelling thermal radiation is found over the tropics and subtropics, lowest values are found in Antarctica.
• With the exception of the inner tropics, there are slight differences between sea and land BOA downwelling thermal radiation in all datasets. Higher values are measured over the ocean than over land. Over land, mountains such as the Himalayas or the Andes are noticeable due to their clearly lower values.
• Similar to the global mean, the temporal variability of CDR datasets also shows a good agreement. In addition, the variability over land is significantly higher than over the ocean, with the exception of a narrow band in the inner tropics. The highest variability is found in East Asia.
• Strong seasonal cycles are visible with higher values in boreal summer and lower values in boreal winter. CDR, as well as all Cloud_cci datasets, time series also contain a small positive trend.

2.3 Surface Reflected Shortwave Radiation (SRS)

The TCDR accuracy of SRS is estimated from the accuracy of the Surface Incoming Shortwave radiation (SIS) and the Surface albedo (SAL).

Applying the error propagation, the accuracy of the SRS product can be estimated as:

\Delta SRS= \frac{\delta SRS}{\delta SIS} \Delta SIS + \frac{\delta SRS}{\delta SAL} \Delta SAL = SAL \Delta SIS + SIS \Delta SAL, \quad  (Eq. 1)

Where

\Delta SIS

comes from [D1] and DSAL is considered as 25% of the SAL value. SAL is estimated as the ratio between Surface Reflected Shortwave radiation (SRS) and Surface Incoming Shortwave radiation (SIS).

SAL = SRS / SIS, \quad  (Eq. 2)

The resulting global mean accuracy for the TCDR SRS is 4.6 W/m².

Accuracy of ICDR SRS is -1.4 W/m² and is estimated in comparison with CERES (section 2.2).

2.4 Surface Net Shortwave Radiation (SNS)

The Surface Net Shortwave radiation (SNS) is calculated using:

SNS = SIS - SRS, \quad  (Eq. 3)

And the accuracy will be estimated as:

\Delta SNS = \Delta SIS + \Delta SRS, \quad  (Eq. 4)

The resulting global mean accuracy for the SNS is 13 W/m² and 3.7 W/m² for TCDR and ICDR respectively.

2.5 Surface Net Longwave Radiation (SNL)

Surface Net Longwave radiation (SNL) is calculated [D2] from:

SNL = SDL - SOL, \quad  (Eq. 5)

The accuracy

\Delta SNL

 will be estimated as:

\Delta SNL = \Delta SDL + \Delta SOL, \quad  (Eq. 6)

The resulting global mean accuracy for the SNL is 23 W/m² for TCDR and 9.4 W/m² for ICDR.

2.6 Surface Radiation Budget (SRB)

The total Surface Radiation Budget (SRB) is the sum of the short and longwave contributions:

SRB = SNS + SNL, \quad  (Eq. 7)

The accuracy will be estimated as:

\Delta SRB = \Delta SNS + \Delta SNL, \quad  (Eq. 8)

The resulting global mean accuracy for the SRB is 36 W/m² for TCDR and 13 W/m² for ICDR.

3. Application(s) specific assessment

...

4. Compliance with user requirements

There are no direct user requirements for the Surface Radiation Budget defined in the Cloud_cci project. Looking at the GCOS ECV requirements for Surface Radiation Budget (https://gcos.wmo.int/en/essential-climate-variables/surface-radiation/ecv-requirements) the values for SIS and SDL are 1 W/m² uncertainty, while the TCDR dataset achieves an accuracy of 8 W/m² for SIS and 12 W/m² for SDL.

ICDR accuracies (estimated with the first 24 months only) appear to be better than TCDR accuracies. ICDR accuracies have been estimated in comparison with CERES surface fluxes.  The better agreement of the ICDR dataset with CERES could come from different factors: (i) comparison of satellite vs satellite instead of satellite vs ground measurements; (ii) wider swath of SLSTR measurements; (iii) closest assumption (auxiliary data) used by both CERES and C3S estimate in the period considered (SRS and SOL strongly depend on surface reflectance and emissivity)   

Table 3 provides an overview of the GCOS requirements for the surface radiative balance and the values achieved by the TCDR parameters. ICDR accuracies, estimated with 24 months of comparison with similar satellite dataset, show consistent results with TCDR. It should be noted that GCOS requirements are targets and are often not attainable using existing or historical observing systems. The Cloud_cci doesn’t meet the requirement for resolving the diurnal cycle due to the nature of the satellite observations, but exceeds the spatial resolution.

Known limitations [From D1 table 7.1]:

• Higher uncertainties in twilight conditions, especially in the shortwave fluxes, due to limitation in retrieving cloud optical thickness and effective radius (input to the radiation calculation) in these condition
• Partly spare temporal/spatial sampling (partly compensated by introduced diurnal cycles correction).
• Downwelling longwave fluxes seem biased high.

...

Variable

...

GCOS Targets

...

Cloud_cci accuracy and stability

...

SIS

...

SDL

...

Loeb, N.G., Doelling, D.R., Wang, H., Su, W., Nguyen, C., Corbett, J.G., Liang, L., Mitrescu, C., Rose, F.G., and Kato, S.: Clouds and the Earth's Radiant Energy System (CERES) Energy Balanced and Filled (EBAF) Top-of-Atmosphere (TOA) Edition 4.0 Data Product, J.Climate, 31(2), 895–918, https://doi.org/10.1175/JCLI-D-17-0208.1.

Poulsen, C. A., McGarragh, G. R., Thomas, G. E., Stengel, M., Christensen, M. W., Povey, A. C., Proud, S. R., Carboni, E., Hollmann, R., and Grainger, R. G.: Cloud_cci ATSR-2 and AATSR data set version 3: a 17-year climatology of global cloud and radiation properties, Earth Syst. Sci. Data, 12, 2121–2135, 2020, https://doi.org/10.5194/essd-12-2121-2020.

Stengel, M., Stapelberg, S., Schlundt, C., Karlsson, K.-G., Meirink, J.F., Poulsen, C., Bojanowski, J., and Stöckli, R.: Cloud_cci Product validation and Intercomparison Report, http://www.esa-cloud-cci.org/sites/default/files/upload/Cloud_cci_D4.1_PVIR_v6.1.pdf.

...