Last modified on Apr 24, 2023 14:42

Contributors: O. Bobryshev (DWD), A. C. Mikalsen (DWD), G. Thomas (STFC-RAL)

Issued by: SMHI/Karl-Göran Karlsson

Date: 19/02/2021

Ref: C3S_D1.6.1-2020_202012_TRGAD_SRB_v1

Official reference number service contract: 2018/C3S_312b_Lot1_DWD/SC1

History of modifications

Related documents

Acronyms

General definitions

The meaning of the terms uncertainty, accuracy and error is often difficult to interpret and may be treated differently in various referred documents. In this document we adopt the following interpretation:
The accuracy, uncertainty or error of an estimated ECV (or, more formally, Thematic Climate Data Record, TCDR) is described by three differently contributing components:

1. The systematic error
2. The random error
3. The time-dependent error

The systematic error is commonly the mean error or the Bias. For non-Gaussian distributions of the error the median or the mean absolute error can be a more useful quantity.
The random error is commonly the root-mean-squared deviation RMSD. Sometimes the Bias is subtracted yielding the centered root-mean-squared deviation cRMSD. Notice that if the Bias is zero the two mentioned quantities are equal and may be interpreted as the standard deviation of the error (often denoted standard error).
The time-dependent error is commonly the change in Bias over time (for ECVs or TCDRs over decades). We call this parameter stability.

All TCDRs are normally evaluated against target requirements for the systematic, random and time-dependent error.
This C3S project also deals with extensions of TCDRs, i.e. products derived from continued processing of the CDRs using the same methods and algorithms as originally used for TCDR production. We denote these CDRs Intermediate Climate Data Records, ICDRs. To evaluate the ICDR compliance with original TCDRs, a different approach in terms of defined requirements is followed. The ICDR is assessed on the basis of the TCDR distribution with respect to a reference validation source. After calculation of this distribution of differences, the ICDR is evaluated against the same reference and a binomial test is applied to verify that 95 % of the difference values lie within the upper and lower bounds of the TCDR difference distribution. The lower and upper bounds of the difference distribution is defined as the 2.5^th and 97.5^th percentiles of the difference distribution.

More details on the estimation of errors and uncertainty parameters are given in the Report on Updated KPIs' [D9].

Scope of the document

This document provides relevant information on requirements and gaps for two Surface Radiation Budget products based on AVHRR data and one product based on (A)ATSR and SLSTR data. The first AVHRR-based data record is the CLARA-A2.1 data record (CLARA-A2: CM SAF CLoud, Albedo and Radiation data record – AVHRR based, Edition 2.1) and the second is the CLARA-A2.1 extra data products and their respective ICDR extensions. The (A)ATSR-based data record is produced in the ESA CLOUD-CCI project and it is extended with SLSTR based products generated specifically for C3S.

The document is divided into three parts. Part 1 describes the product the present document refers to. Part 2 provides the target requirements for the product. Part 3 provides a past, present, and future gap analysis for the product and covers both gaps in the data availability and scientific gaps that could be addressed by further research activities (outside C3S).

Executive summary

The Surface Radiation Budget products are described together with their target requirements. They include three data records:

• The CLARA-A2.1 data record (CLARA-A2: CM SAF CLoud, Albedo and Radiation data record – AVHRR based, Edition 2.1),
• The CLARA-A2.1 extra data products and their respective ICDR extensions
• The ESA Cloud_cci – (A)ATSR based v3.0 data record and its SLSTR-based ICDR extension.

The CLARA-A2.1 data record is the core data record for AVHRR-based data. The extra data products fluxes are not included in the brokered CLARA-A2.1 dataset and are calculated specifically within the C3S project as complimentary data for the convenience of users. This separation into two datasets is necessary to keep the origin of the data, e.g. licence affiliations: "EUMETSAT's CM SAF" and "Copernicus", clear for individual products (see the following table).

Licence overview of the CLARA product family Surface Radiation Products

Please note, that in contrast to the original CM SAF CLARA-A2.1 dataset, the brokered dataset only provides:

• Level-3 data – excluding the level-2b data format.
• Data on a global equal angle grid – excluding the polar grid format.
• An aggregated version of all satellite data – excluding the provision of the individual satellite datasets

The target requirements (expressed as mean absolute error) for the AVHRR-based monthly mean products (from CLARA-A2.1 and extra data products) are set to 10 Wm^-2 for SIS, SOL and SDL. SIS is also provided as daily means with a target accuracy of 20 W m^-2. Stability requirements are set to 2 Wm^-2dec^-1 for SIS (also for daily means) and to 3 Wm^-2dec^-1 for both SDL and SOL. Many features of the CLARA-A2.1 extra data products are similar to the ones of CLARA-A2.1, and there are no formal requirements for their accuracy and stability. Their validation approach is based on the method of error propagation.

The ESA Cloud_cci surface radiation budget data record and its SLSTR-based ICDR extension are described, along with their target requirements and key performance indicators.
For the ESA Cloud_cci data record, the target requirements are defined by GCOS, and are defined as an accuracy of 1 Wm^-2 for the monthly, global mean surface radiation budget in both shortwave and longwave. Stability requirements are 0.2 Wm^-2dec^-1 for both short and longwave.

The data record of the (A)ATSR sensors runs from 1995-2012. The SLSTR-based ICDR data record starts in 2017. Options for filling the four-year gap between the end of the AATSR data record and the beginning of the SLSTR record are also provided. A key advantage of the (A)ATSR and SLSTR data (compared to AVHRR) is that the data are highly stable, both in terms of satellite orbital parameters and instrument calibration, at the cost of temporal coverage and instrument swath width.

An extensive description of past, current and future availability of data from the Advanced Very High Resolution Radiometer (AVHRR) and the (A)ATSR + SLSTR data records is given. In addition, future prospects of utilizing AVHRR-heritage spectral channel data from new imaging sensors on new satellites are described.

It is concluded that the AVHRR-based observations series, based on one morning and one afternoon orbit constellation, can be prolonged to reach at least 60-year duration if adding AVHRR-heritage information. However, for this to become realized, efforts are needed to harmonize and homogenize observations between true AVHRR data and AVHRR-heritage data. This concerns both calibration aspects and spatial resolution aspects. Further developments of surface radiation products are required in particular over bright surfaces, i.e., deserts and polar snow- and ice-covered areas. Work is also needed for a better characterization of conditions with high solar zenith angles and for improved estimates of fluxes under cloudy conditions. A future continuation of active observations from space is judged as crucial for further development of retrieval methods based on AVHRR-heritage data.

Regarding the future availability of SLSTR data, the goal of the Copernicus Sentinel satellite program (jointly funded by ESA and EU) is to provide high-quality and sustained measurements for climate and environmental monitoring. Consequently, a measurement prolongation beyond the current Sentinel-3A and Sentinal-3B satellites is planned for with two more satellites (Sentinel 3C and Sentinel 3D).

1. Product description

1.1 AVHRR-based Surface Radiation Products

1.1.1 Licensing overview for the AVHRR-based Surface Radiation Budget Products

Surface Radiation Budget Products brokered and produced up-to-date within C3S_312b Lot1 come from two main sources: brokered from EUMETSAT's CM SAF and produced within the C3S project itself. Table 1-1 shows the comprehensive overview of all current and future datasets and licence owners. This separation into two datasets is necessary to keep the origin of the data, e.g. licence affiliations: "EUMETSAT's CM SAF" and "Copernicus", clear for individual products.

For the purposes of this document, we will combine two data records: Surface Radiation Budget AVHRR CLARA (1) TCDR v2.0+ ICDR v2.x (known as brokered EUMETSAT's CM SAF CLARA-A2.1) and the Surface Radiation Budget AVHRR (2) TCDR v2.0 + ICDR v2.x (known as CLARA-A2.1-extra data products). The two data records are combined into one group called the "CLARA product family" to which CM SAF's CLARA-A2.1 makes a major contribution.

Table 1-1: Licence overview of the CLARA product family Surface Radiation Products

The extra data products are not included in the CLARA-A2.1 dataset and are calculated specifically within the C3S project as complimentary data for the convenience of users. Furthermore, combining all variables in a "CLARA product family" group logically combines all variables that are needed for the full description of the Energy Budget.

The longwave fluxes are not included in the CM SAF ICDR plans for the current phase. To ensure the dataset integrity and continuity, they are calculated within the C3S for the ICDR part using algorithms developed by CM SAF. As such, the longwave fluxes change their licence affiliation, namely they are provided within the C3S project for the ICDR part (2019-2020) and are brokered from EUMETSAT's CM SAF for the TCDR part (1982 to 2018).

1.1.2 CLARA-product family

The CLARA-A2.1 record comprises 37 years (January 1982 - December 2018) of satellite-based climate data records derived from the measurements of the Advanced Very High Resolution Radiometer (AVHRR) onboard the polar orbiting NOAA and METOP satellites. ICDR production covers data from January 2019 to December 2020. Table 1-2 shows an overview of successive AVHRR sensors onboard dedicated platforms that are included in the generation of the CLARA-A2.1 dataset. The brokered CLARA-A2.1 datasets are provided on a regular global latitude-longitude grid with a spatial resolution of 0.25° x 0.25°.

Table 1-2: Different versions and the lifetime of AVHRR on different platforms

The retrieval of the shortwave surface radiation parameters is based on the atmospheric transmission and the associated reflected irradiance at the surface, by the satellites. The retrieval of the longwave surface radiation parameters is based on the cloud information from CLARA-A2.1 cloud products, reanalysis data and topographic information. Generation of the extra data products benefits greatly from the EUMETSAT's CM SAF cLoud, Albedo and surface RAdiation dataset from AVHRR data - Edition 2 (CLARA-A2.1 dataset. Therefore, many features of the dataset are similar to the ones of CLARA-A2.1.

Omitting complicated license described in the previous section, the CLARA Product family consists of seven three products: the Surface Incoming Shortwave radiation (SIS), the Surface Outgoing Longwave radiation (SOL), and the Surface Downwelling Longwave radiation (SDL), the Surface Reflected Shortwave radiation (SRS), the Surface Net Shortwave radiation (SNS), the Surface Net Longwave radiation (SNL), and the Surface Radiation Budget (SRB). All products are provided as monthly means, and SIS as monthly and daily means. Table 1-3 shows association between legacy CLARA product names and CF Standard names used in the Climate Data Store (CDS). The CLARA names are included in the netCDF files as "long_names", and CF names as "standard_names",

Table 1-3: Association table of CLARA product names and CF Standard names used in the CDS (Climate Data Store)

AVHRR Global Area Coverage (AVHRR GAC) data is the fundamental climate data record used for generation of the Surface Radiation Budget datasets. The detailed description of the algorithm used to generate AVHRR GAC is given in CM SAF ATBD Cloud Products [D5], Section 4.3. Further information on the specific input and auxiliary data can be found in the Algorithm Theoretical Basis Document for this dataset.

1.1.2.1 Surface Incoming Shortwave Radiation (SIS)

AVHRR Global Area Coverage (AVHRR GAC) data are the basis for generation of the surface radiation budget datasets. The detailed description of the algorithm used to generate the required fundamental climate data record (FCDR) for AVHRR GAC is given in CM SAF ATBD Cloud Products [D5], Section 4.3. The set of input and auxiliary data used to generate SIS are given in CM SAF ATBD [D1], Section 2.1.1.x. A flow-chart that summarizes all processing steps is given in CM SAF ATBD [D1], Figures 2-1 and 2-2.

Under the conditions of snow-covered surface and in regions with varying surface albedo, the surface incoming radiation dataset performs poorly in comparison to the reference datasets. Grid points in such areas are masked out. Thus, no data is available for large parts of the polar regions. Another aspect of the algorithm to derive the monthly averages is that it requires a minimum of 20 observations per month. In the period 1982–1991, when only afternoon satellites with AVHRR instruments were operational, some parts of the globe did not have the required number of observations per months, resulting in gaps in the spatial coverage. Moreover, in the case of instrumental errors or satellite transitions, the period from 1982 to 2003 is influenced by measurement gaps of one or more AVHRR instruments, resulting in a reduced accuracy estimation of daily and monthly radiation fluxes.

A full list of the known limitations and their implications for SIS are described in CM SAF ATBD [D1] Chapter 2.1.1.4. In sum, these are:

• Limitation due to the temporal resolution of surface albedo
• Uncertainty under cloudy conditions, especially with thin clouds
• Uncertainties in the cloud-detection algorithm
• Application of monthly climatological aerosol information instead of the real time data

1.1.2.2 Surface Downwelling Longwave Radiation (SDL)

The algorithm used to generate the SDL dataset is described in CM SAF ATBD [D1] Chapter 2.2.1.

The known limitations and their implications for SDL are described in CM SAF ATBD [D1] Chapter 2.2.1.3. These are:

• Underestimation of inter-annual cloud cover variability due to linear regression
• Limited topographic correction by using constant values

1.1.2.3 Surface Outgoing Longwave Radiation (SOL)

The algorithm used to generate SOL dataset is described in CM SAF ATBD [D1] Chapter 2.2.2.

The known limitations and their implications for SOL are described in CM SAF ATBD [D1] Chapter 2.2.2.1. These are:

• Assumption of dry-adiabatic temperature gradient for topographic correction
• Predefined emissivity depending on land type classes ignoring soil moisture

1.1.2.4 Surface Reflected Shortwave Radiation (SRS)

The SRS product is generated from the Surface Incoming Shortwave radiation (SIS) from the Surface Radiation Budget brokered from EUMETSAT’s CM SAF CLARA-A2.1 [D10] and the Surface Albedo (SAL) from the CLARA-A2.1 dataset (monthly means).

The set of input and auxiliary data used to generate the SIS is given in CM SAF ATBD Surface Radiation Products [D1], Section 2.1.1. The set of input and auxiliary data used to generate the SAL is given in CM SAF ATBD Surface Albedo [D11], Section 5.1.

1.1.2.5 Surface Net Shortwave Radiation (SNS)

The SNS product is generated from the SIS and the SAL monthly means.

The set of input and auxiliary data used to generate the SIS is given in CM SAF ATBD Surface Radiation Products [D1], Section 2.1.1. The set of the input and auxiliary data used to generate the SAL is given in CM SAF ATBD Surface Albedo [D11], Section 5.1.

1.1.2.6 Surface Net Longwave Radiation (SNL)

The SNL product is generated from the Surface Downwelling Longwave radiation (SDL) and the Surface Outgoing Longwave radiation (SOL) brokered from EUMETSAT’s CM SAF CLARA-A2.1 [D10].

The set of input and auxiliary data used to generate the SDL is given in CM SAF ATBD Surface Radiation Products [D1], Section 2.2.1. The set of input and auxiliary data used to generate the SOL is given in CM SAF ATBD Surface Radiation Products [D1], Section 2.2.2.

1.1.2.7 Surface Radiation Budget (SRB)

The SRB product is composed as the sum of SNL and SRB.

1.2 (A)ATSR-based Surface Radiation Products from the ESA-CLOUD-CCI project and its SLSTR-based ICDR extension

This section describes the cloud products of the Cloud_cci (A)ATSR v3.0 data record, which we will refer to as Cloud_cci v3, and its extension with the same retrieval scheme applied to the Sea and Land Surface Radiometer (SLSTR), which is produced specifically for C3S.

The Cloud_cci v3 data record is based on Along-Track Scanning Radiometer 2 (ATSR-2) and Advanced A Along-Track Scanning Radiometer (AATSR) observations onboard the ESA 2^nd European Research Satellite (ERS-2) and ENVISAT satellites. Together, the data record provided by these two instruments is often abbreviated to (A)ATSR. The SLSTR instrument, which is the successor to (A)ATSR, is on board the Copernicus Sentinel-3 platform [D13].

Observations are available on a 1x1 km grid, which closely matches the true instrument spatial resolution globally and the final CDR is compiled in a regular global grid with 0.5° latitude-longitude resolution for monthly averages. The covered time period of the Cloud_cci data record ranges from June 1995 to April 2012 and the SLSTR extension provides coverage from January 2017; i.e. there is an almost seven-year gap between the TCDR and corresponding ICDR.

Cloud_cci products were based on the third reprocessing of the AATSR-multimission archive, which included vicarious calibration of the shortwave channels over the entire data record to correct for long-term calibration drift (Smith, 2012). SLSTR products are based on collection 3 of the “non-time critical” SLSTR level 1 archive.

The Cloud_cci v3 data record was produced using the Community Cloud four Climate (CC4Cl) processing chain, which is based around the Optimal Retrieval of Aerosol and Cloud (ORAC) retrieval scheme, both of which are described in detail in [D14, D15] and by Sus et al. (2018) and McGarragh et al. (2018).

In addition to being based on (A)ATSR radiances, the Cloud_cci CC4Cl processing chain makes use of the following auxiliary datasets:

• USGS Digital Elevation Map (USGS, 1996)
• ERA-Interim surface and atmospheric profile temperatures and pressure (Dee et al., 2011)
• ERA-Interim profiles of moisture content and Ozone concentrations (Dee et al., 2011)
• ERA-Interim snow depth and albedo (Dee et al., 2011)
• National Snow and Ice Data Center Near-real-time Ice and Snow Extent (NISE) sea ice concentration (Brodzik and Stewart, 2016).
• ERA-Interim 10 m u and v wind components (Dee et al., 2011).
• MODIS-based land surface bidirectional reflectance distribution function (BRDF) data (MCD43C1 Collection 6, (Schaaf and Wang, 2015)).
• Land surface emissivity from the Cooperative Institute for Meteorological Satellite Studies (CIMSS) “Baseline Fit” database.
• Solar and Heliospheric Observatory (SOHO) and Solar Radiation and Climate Experiment (SORCE) incoming total solar irradiance.

For L1 data outside the temporal coverage of the above auxillary datasets (for example, ATSR-2 data prior to the 1999 launch of MODIS products), climatologies based on the above auxillary datasets are used. In the case of the SLSTR ICDR, ERA-5 data is used rather than ERA-Interim.

The seven surface radiation parameters provided to C3S from the Cloud_cci v3 dataset and SLSTR are the same as those provided by the AVHRR products described in section 1.1.2 and suffer from similar limitations.

2. User Requirements

2.1 CLARA product family

2.1.1 Summary of target requirements

This report covers the following products from CLARA family: the SIS, the SDL, the SOL, the SRS, the SNS, the SNL and the SRB. The predefined requirements for the SIS, the SOL and the SDL are given in CM SAF Product Requirement Document (PRD) [D4], Annex A. The validation of the surface radiation datasets was conducted against surface measurements from the Baseline Surface Radiation Network (BSRN) (Ohmura et al., 1998).

There are three accuracy categories in the CM SAF PRD document ([D4], Section 5): threshold, target and optimal accuracies. They are defined keeping in mind different target users: operational climate monitoring, global and regional climate modelling and global and regional climate studies, respectively.

Target requirement for accuracy (mean absolute error) are selected as follows:

• SIS monthly means: 10 Wm^-2 (corresponds to target accuracy),
• SIS daily means: 20 Wm^-2 (corresponds to threshold accuracy)
• SDL monthly means: 10 Wm^-2 (corresponds to target accuracy) and
• SOL monthly means: 10 Wm^-2 (corresponds to target accuracy).

Stability requirements for the Surface Radiation Budget TCDR are defined as follows:

• SIS monthly means - 2 Wm^-2dec^-1,
• SIS daily means – 2 Wm^-2dec^-1, and
• SDL and SOL monthly means – 3 Wm^-2dec^-1).

At the time (~2010) when requirements for the CLARA-A2 data record had to be defined in [D4], there was no guidance available for surface radiation products in the available GCOS-107 document [D6]. Instead, requirements had to be set in a dialogue with experts and potential users (e.g., in association with CM SAF User Workshops).

The above-mentioned targets are met by all datasets. The general validation methodology and results are described in more detail in the CM SAF Validation Report [D3], Section 5.1. The validation of the SIS, SOL and SDL products are described in the CM SAF Validation Report [D3], Sections 5.2, 5.3 and 5.4 respectively.

The SRS, the SNS, the SNL and the SRB products (i.e., products calculated specifically for the C3S project) are validated using the method of error propagation. The general validation methodology and results are described in more detail in the C3S PQAR [D12], Sections 1.1-1.4. Table 2-1 provides a summary of the calculated accuracies for the SRS, the SNS, the SNL, and the SRB.

Table 2‑1: Summary of the accuracy of the CLARA-A2.1 extra data products

2.1.2 Key Performance Indicators - KPIs

Since the CLARA-A2.1 surface radiation budget products are brokered from the CM SAF project, and consequently cannot be altered in C3S_312b_Lot1, the current target requirements on the key performance of the data set, measured within C3S by the so-called Key Performance Indicators (KPIs), are defined as the achieved requirements (with original requirements described in [D4]) in previous CLARA-A2.1 validation activities in CM SAF. These values are listed in section 2.1.1.

For the evaluation of the ICDR, corresponding products from the ground-based BSRN measurements are used as reference. The distribution of the global differences between BSRN products and the CLARA-A2.1 TCDR has been compiled and the corresponding 2.5 and 97.5 percentile differences are given in Table 2-2.

These percentiles are used to check, by means of a binomial test at 5 % significance level, whether the corresponding ICDR differences are consistent with the TCDR differences or not. Further details on these tests are found in the Report on Updated KPIs [D9].

Table 2‑2: Key Performance Indicators (KPIs) for the surface radiation budget products of interest for C3S_312b_Lot1

2.1.3 Discussion of requirements with respect to GCOS and other requirements

The product requirements listed in the CM SAF Product Requirement Document [D4] for the CLARA-A2.1 data record were generally defined in accordance with the GCOS report GCOS-107 [D6]. However, since this version of the GCOS document did not yet include requirements for the Surface Radiation ECV, some CM SAF-specific requirements had to be defined and used (as explained above in section 2.1.1).

New GCOS requirements for the ECV Surface Radiation Budget are summarized in GCOS-200 [D7] and include requirements for the horizontal resolution, temporal resolution, accuracy and stability. However, these requirements are only valid for the net fluxes (i.e., SNS and SNL) and not for all individual radiation budget components (see Table 2-3 below in relation). However, one could claim that individual radiation budget components should consequently be constrained in the same way as net fluxes.

All products in the brokered CLARA-A2.1 dataset fulfil the new GCOS requirements regarding the horizontal and temporal resolution. However, the SNS and SNL products do not fulfil the new requirements on accuracy and stability which are very stringent compared with previously used requirements for the CLARA data record. Nevertheless, achieved accuracy and stability results allow consistent quantification of mean values, anomalies, variability and the Earth energy budget in general. We point out the existing uncertainties in the methodology of comparison with area-to-point measurements (i.e. satellite-area to point-ground-based reference networks) as important reasons for not fulfilling the new requirements.

2.1.4 Data format and content issues

Information on the file format is provided in the CM SAF Product User Manual (PUM) [D2] Section 4. The CLARA A-2.1 surface radiation products are defined using standard data formats (netCDF4) and map projections (regular latitude/longitude). Meta data and naming definitions follow the Climate & Forecast (CF) conventions (https://cfconventions.org).

Based on the recommendations formulated within C3S_312b Lot1, the license field was added to all the netCDF-files that are brokered and produced. The goal is to provide a straightforward identification of the data record producer.

2.2 (A)ATSR-based Surface Radiation Products from the ESA-CLOUD-CCI project

2.2.1 Summary of target requirements

The target requirements for the Cloud_cci products are defined by the WMO Global Climate Observing System (GCOS) initiative, which defines and lays down targets for the observation of ECVs (Table 2‑3).

Table 2‑3: Target requirements for surface radiation budget defined by GCOS-200 (document [D7], Table 23, page 279].

The Cloud_cci product, and the SLSTR extension, achieve or exceed the frequency and resolution requirements, with the exception of resolving the diurnal cycle, which is not possible with a single low-Earth-orbit platform. The GCOS accuracy target of 1 Wm-2 is not met by the cloud_cci/SLSTR products, however it should be noted that GCOS requirements are targets for what should be achievable through Earth observation and are often not attainable using existing or historical observing systems.
GCOS only defines targets for the net-radiation fluxes, but as these are simple sums of the up- and down-welling fluxes, they provide target constraints for the entire suite of surface radiation parameters provided by the Cloud_cci products.

2.2.2 Key Performance Indicators - KPIs

Similar to the CLARA-A2.1 products, the Cloud_cci (A)ATSR products are brokered products (but in this case from the ESA CCI programme) and cannot be altered within the scope of C3S_312b_Lot1. However, this dataset forms the basis of the KPIs for the SLSTR based ICDR going forward. The KPIs for the SRB product are based on comparison against the NASA Clouds and the Earth's Radiant Energy System (CERES) instruments. These comparisons are represented as the 2.5 and 97.5 percentiles of the distribution of differences between (A)ATSR or SLSTR monthly-mean values and the corresponding CERES values (corrected for the mean seasonal cycle). These values, calculated from the 12-year (A)ATSR CDR are summarized in Table 2-4.

It should be noted that CERES surface radiation values are a product of a similar level of processing, based on knowledge or assumptions of the atmospheric state, as those produced from the CC4Cl retrieval scheme. Thus, CERES should not be considered as a more accurate estimate of SRB than the Cloud_cci CDR. However, significant effort has gone into ensuring the stability and consistency of the CERES products, making it suitable for monitoring the relative performance of the Cloud_cci products and their extension with SLSTR.

Table 2-4: Key performance indicators (KPIs) for the Cloud_cci SRB record, to be applied to the SLSTR ICDR data.

2.2.3 Discussion of requirements with respect to GCOS and other requirements

A discussion on these requirements has already been provided in section 2.2.1 (related to Table 2-3).

2.2.4 Data format and content issues

The Cloud_cci v3 cloud property products are defined using standard data formats (netCDF) and map projections (regular latitude/longitude grids). Meta data definitions follow the Climate & Forecast conventions (http://cfconventions.org/).

3. Gap Analysis

3.1 Description of past, current and future satellite coverage

3.1.1 CLARA product family

The surface radiation and cloud products belong to the CLARA-A2.1 datasets. These two datasets make use of the same instruments, installed on the same satellites. A full presentation of the AVHRR sensor and relevant satellites can be found in the C3S_D1.6.1-2020_202012_TR_GA_v1.0, Section 3.1.1 Cloud properties TCDR AVHRR CLARA v2.0 + ICDR v2.x [D8].

Known data gaps

Monthly dataset:

• February 1985

Daily dataset:

• 1982-05-29–1982-05-31
• 1982-09-25–1982-09-26
• 1983-07-27–1983-08-02
• 1983-08-06
• 1983-09-21–1983-09-26
• 1984-01-14–1984-01-15
• 1984-07-23
• 1984-12-06
• 1985-02-03–1985-02-24
• 1986-03-15

3.1.2 (A)ATSR-based Surface Radiation Products and its SLSTR-based ICDR extension

The Cloud_cci v3 is based on radiances provided by the ATSR series of sensors. These instruments flew on sun-synchronous polar orbiting satellites with daytime equatorial crossing times in the mid-morning; 10:30 Local Time on Descending Node (LTDN) for ATSR-2 and 10:00 LTDN for ENVISAT, with both satellites sharing the same ground track. There were 14.3 orbits per day, meaning 28 equatorial overpasses per-day, with measurements covering a total of 18% of equatorial circumference of the Earth (with equally spaced 512 km swaths). The observation frequency increased at higher latitudes (with a maximum of 14 observations per day at the poles) due to increasing overlaps between the satellite swaths. Both sensors provided the same seven channels (and used the same conical dual-viewing geometry), but not all channels were provided at all times, or at full digitization rate, from ATSR-2, due to limitations of the data bandwidth provided by the ERS-2 platform. Over ocean regions, ATRS visible channels were often only provided in a 256 pixel "narrow-swath" mode. The channels provided by both instruments were centered at 0.55, 0.67, 0.87, 1.6, 3.7, 10.8, 12.0 m and the filter band passes were very similar between instruments. Despite the low-data rate modes of ATSR-2, the combination of the very similar instrument specifications, very close orbital parameters and the lack of any significant orbital drift in the ERS-2 and ENVISAT satellites mean that ATSR-2 and AATSR provide a highly consistent data record, especially when compared to that provided by the AVHRR record used by the CLARA-A2 TCDR (although AVHRR provides a much longer data record).

The TCDR from Cloud_cci v3 begins with the launch of ERS-2 in mid-1995 and continues until the failure of ENVISAT in April 2012. Due to instrument problems, there is a six-month data gap in the ATSR-2 record from January to June 1996.

There is an overlap of 1 year of data between the two platforms, between mid-2002 (when ENVISAT was launched) and mid-2003 (when the onboard data storage on ERS-2 failed). There is additional ATSR-2 data available up-to 2009, but this is not global as data could only be collected when the satellite was within line-of-sight with a ground receiving station, and has not been included in the TCDR. There is some scope to push the coverage of the ATSR cloud record back to 1991, by using the ATSR-1 instrument (onboard ERS-1), which also flew in a similar orbit to its successors. However, ATSR-1 lacked the shortwave channels (apart from the 1.6 m) channel, which would reduce the information available to daylight retrievals and would represent a significant inhomogeneity in the TCDR.

The extension of the ATSR TCDR makes use of SLSTR sensors onboard the Sentinel-3 platforms. SLSTR represents a significant upgrade over (A)ATSR, providing a wider swath, two satellites within interleaved orbit swaths, additional channels and the data available security of an operational system. The Sentinel-3s have a very similar orbit to ENVISAT and the ERS satellites, with a sun-synchronous orbit with an LTDN of 10:00, and 14.3 orbits per-day. However, there is slightly over a 4-year gap between the end of the AATSR record and the first SLSTR data. There are several options available to fill this gap, as ORAC can be applied to most radiometers with similar channels to those provided by ATSR. Indeed, cloud CDRs of ORAC applied to both MODIS and AVHRR already exist, having been produced in the Cloud_cci program, but have not been brokered to the CDS.

It should also be noted that ORAC could also be applied to the VIIRS and MetImage instruments described above, which could complement the SLSTR ICDR.

Regarding the future availability of SLSTR data, the goal of the Copernicus Sentinel satellite program (jointly funded by ESA and EU) is to provide high-quality and sustained measurements for climate and environmental monitoring. Consequently, a measurement prolongation beyond the current Sentinel-3A and Sentinal-3B satellites is planned for with two more satellites (Sentinel 3C and Sentinel 3D).

3.2 Development of processing algorithms

The surface radiation and cloud products belong to the CLARA-A2.1 TCDR and use the same satellite sensors from the AVHRR-family. Thus, the pre-processing methods are the same for these two datasets, such as insuring the quality checks and corrections of the radiances and a homogeneity of the input data series. Full account of these issues can be found in the C3S_D1.6.1-2020_202012_TR_GA_v1.0, Section 3.2.1 Cloud properties TCDR AVHRR CLARA v2.0 + ICDR v2.x [D8]. Specific issues and research needs of the surface radiation datasets are described later in the section 3.5.

3.2.1 CLARA product family

Surface Downwelling Longwave and Surface Outgoing Longwave Radiation use the ERA-Interim data on the surface downwelling longwave fluxes and surface temperature, respectively. When ERA-Interim was discontinued in 2019, operational analyses from the ECMWF IFS model was used as replacement. This might have caused a transition jump followed by a constant offset in the data but this has not yet been quantified. However, only the ICDR AVHRR CLARA v2.0 product is affected by this.

3.2.2 (A)ATSR-based Surface Radiation Products and its SLSTR-based ICDR extension

As with the CLARA CDRs, the stability and quality of the input data is the key parameter which influences the reliability of the Cloud_cci v3 CDRs. The (A)ATSR TCDR is based on version 3 of the "AATSR multimission archive" maintained by CEDA and the UK National Earth Observation Data Centre (NEODC). This record incorporates the latest calibration corrections (including long-term drift corrections from vicarious calibration) and represents the most consistent and accurate record of radiances from the (A)ATSR record. A future update to this record would make a reprocessing of the Cloud_cci TCDR possible.

In the case of the SLSTR ICDR, the status of the level 1 radiances is considerably less stable. Data from early in the SLSTR record has considerably worse calibration and geolocation than more recent data. When a fully reprocessed version of the data record becomes available in the future, regeneration of the cloud ICDR would be possible.

EUMETSAT has provided updated calibration corrections to SLSTR shortwave channels, communicated through the Sentinel-3 Scientific Validation Team (S3VT), which have been applied retrospectively. However, there is not yet any information on the stability of the SLSTR calibration over time and there remain issues with the colocation of SLSTR channels in early versions of the level 1 products.

3.2.2.1 Adaptions of the ORAC scheme to better exploit SLSTR

As mentioned above, SLSTR provides some additional channels over the earlier AATSR instruments. Of particular note is the new 1.3 m channel, which, due to its location in a water-vapour absorption feature, is particularly sensitive to the presence of high-altitude clouds. Utilizing this channel in the retrieval scheme itself is unlikely to be beneficial, as accurate knowledge of the water vapour profile is needed to accurately model the radiances. However, the use of this channel in prior cloud-detection and characterization is to be investigated in coming Cloud_cci+ work.

3.2.2.2 Forward model improvements

Further improvements to the forward modelling of clouds for the ORAC retrieval scheme are also underway. In particular:

• The SLSTR ICDR makes use of ERA-5, rather than the ERA-Interim used for the TCDR.
• The spectral dependence of cloud scattering and absorption will be modelled across the bandpass of the instrument channels (rather than at the channel centre as previously).
• At present cloud is modelled as an infinitesimally thin layer within an atmosphere modelled by RTTOV. The modelling of cloud geometric thickness effects will also be investigated in the upcoming Cloud_cci+ project.
• The use of new ice cloud optical properties will also be investigated, as these become available.
• Improvements in the propagation of uncertainty from L2 products to gridded L3 products is also under investigation.

3.3 Methods for estimating uncertainties

3.3.1 CLARA product family

The current CLARA-A2.1 products are not associated with any uncertainty estimates. Thus, uncertainty information is only available as results achieved by associated validation activities [D3].

3.3.2 (A)ATSR-based Surface Radiation Products and its SLSTR-based ICDR extension

The ORAC retrieval scheme provides propagated uncertainties on the retrieved cloud parameters, but these are not propagated through the broadband flux calculations at present. Thus, no uncertainty estimates are provided in the Cloud_cci SRB products, aside from the standard deviation of the level-2 pixels included in each monthly-mean grid box. Thus, as with CLARA-A2.1 products, uncertainty information is only available through validation activities.

3.4 Opportunities to improve quality and fitness-for-purpose of the CDRs

3.4.1 CLARA product family

The upcoming new edition of the CLARA dataset, CLARA-A3, will cover an extended period from 1979 to 2019. Adding these years to the currently brokered dataset is of great value for the climate studies. CLARA-A3 will also provide improved cloud detection and a revised AVHRR calibration. This will lead to an improved quality of surface radiation products for the entire covered time period.

3.4.2 (A)ATSR-based Surface Radiation Products and its SLSTR-based ICDR extension

Most potential improvements to ORAC radiative flux products stem from improvements to the underlying cloud retrieval scheme, which are discussed below.

The planned development of the ORAC retrieval scheme, as applied to (A)ATSR and SLSTR, has already been described in section 3.2.2. ORAC is under active development, both through the ESA CCI+ program and through national UK funding (in particular, under the National Centre for Earth Observation). New improvements of the scheme, where applicable, will be fed through to the production of improved CDR products from SLSTR.

It is also worth noting that the ORAC scheme is not specifically designed for application to (A)ATSR or SLSTR. CDRs have already been produced using the scheme for the AVHRR and MODIS instruments, under previous iterations of the CCI program. The scheme has also been applied to geo-stationary sensors (SEVIRI, GOES and Himawari-AHI), and improved application of the scheme to SEVIRI in particular (making use of the water-vapor sounding channels provided by the instrument) is being undertaken in CCI+.

The code includes the ability to utilize sounding channels (CO₂ slicing and water-vapor absorption), as well as a multi-layer cloud retrieval mode [D16], which greatly improve on the shortcomings of the existing “heritage channel” (AVHRR-like) CDRs produced in CCI, and retrieves the properties of dual-layer cloud scenes. Thus, the scheme provides the scope for the production of cutting-edge CDRs from a wide range of instruments, all with a consistent retrieval approach.

3.5 Scientific Research needs

3.5.1 CLARA product family

3.5.1.1 Surface Incoming Shortwave Radiation (SIS)

The known issue is poor performance of SIS compared with reference datasets in regions with highly-reflecting surfaces. For this reason, studies in relation to snow detection are needed. Additional studies with respect to satellite observations under twilight/dawn conditions could improve the data quality of prior satellite generations.
Furthermore, retrieval of the SIS under the cloudy conditions requires the broadband shortwave flux estimate. In the current version, this conversion is done using two satellite channels. Further investigations are needed on the effect of such conversion on the retrieved SIS values.

3.5.1.2 Surface Outgoing Longwave Radiation (SOL)

The topographic correction of the SOL from ERA-Interim is based on the assumption that the surface temperatures are modified with the dry-adiabatic temperature gradient as a function of elevation. This assumption needs to be verified.

Another issue for all satellite measurements is the estimate of the surface emissivity. It is desirable to have routinely updated emissivity estimates.

3.5.2 (A)ATSR-based Surface Radiation Products and its SLSTR-based ICDR extension

The requirements for the further improvements of the (A)ATSR and SLSTR CDRs are identical to those for the AVHRR CLARA CDRs.

3.6 Opportunities from exploiting the Sentinels and any other relevant satellite

3.6.1 CLARA product family

Surface Radiation Budget products are brokered without changes. In the current version no information from the Sentinel satellites is used. However, the Surface Radiation Budget ESA SLSTR ICDR v3.x dataset is directly benefiting from the SLSTR instrument installed on the Sentinel-3 series satellites. This instrument has almost identical channels as AVHRR and also some unique channels. This gives a possibility for its use for inter-comparison.

3.6.2 (A)ATSR based Surface Radiation Products and its SLSTR-based ICDR extension

The ESA SLSTR v3.x ICDR directly exploits data from the Sentinel-3 platform. There have been examples shown of utilizing Sentinel-3 OLCI-like measurements (mainly using MERIS on ENVISAT) for cloud retrieval in conjunction with (A)ATSR or SLSTR (Carbajal Henken et al. 2014), but difficulties in cross-calibration and co-registration of the different instruments have meant these products have not shown improved performance over the (A)ATSR/SLSTR only algorithms. The availability of a well co-located and calibrated joint SLSTR-OLCI L1 product, could resurrect this approach to further improving cloud products derived from Sentinel-3 (and the preceding ENVISAT).

As discussed in section 3.4.2, the ORAC retrieval scheme can be, and has been, applied to a wide range of satellite visible-IR imaging radiometers. A particular instrument, of direct relevance to the Sentinel satellite program is the Flexible Combined Imager (FCI) to fly on MeteoSat Third Generation/Sentinel-4. This instrument is essentially a replacement for the SEVIRI sensors on MSG, with capabilities similar to those provided by Himawari-AHI and GOES-ABI imagers (which ORAC has already been applied to).

References

Brodzik, M. J. and Stewart, J. S.: Near-real-time SSM/I-SSMIS EASE-grid daily global ice concentration and snow extent, Version 5, Boulder, Colorado USA. NASA National Snow and Ice Data Center [data set], https://doi.org/10.5067/3KB2JPLFPK3R, 2016.

Carbajal Henken, C.K., R. Lindstrot, R. Preusker and J. Fischer, 2014: FAME-C: cloud property retrieval using synergistic AATSR and MERIS observations. Atmos. Meas. Tech., 7, 3873–3890, doi:10.5194/amt-7-3873-2014.

Dee et al. (35 named authors), 2011: The ERA-Interim reanalysis: configuration and performance of the data assimilation system, Quart. J. Royal Met. Soc., 137 (656), 553 - 597. Online version: http://onlinelibrary.wiley.com/doi/10.1002/qj.828/abstract.

McGarragh, G.R.; Poulsen, C.A.; Thomas, G.E.; Povey, A.C.; Sus, O.; Stapelberg, S.; Schlundt, C.; Proud, S.; Christensen, M.W.; Stengel, M.; et al., 2018: The Community Cloud retrieval for CLimate (CC4CL). Part II: The optimal estimation approach. Atmos. Meas. Tech., 11, 3397–3431.

Ohmura, A., et al. (1998), Baseline Surface Radiation Network (BSRN/WCRP): New precision radiometry for climate research, Bulletin of the American Meteorological Society, 79(10), 2115-2136.

Smith D. 2012: ENVISAT AATSR Instrument Performance – End of Mission Report. Report, accessed 02-Mar-2020:
https://atsrsensors.org/pdf/ENVISAT%20AATSR%20Instrument%20Performance%20-%20End%20of%20Mission%20Report%20%20-%20Issue%201.0.pdf

Sus, O.; Stengel, M.; Stapelberg, S.; McGarragh, G.; Poulsen, C.; Povey, A.C.; Schlundt, C.; Thomas, G.; Christensen, M.; Proud, S.; et al., 2018: The Community Cloud retrieval for Climate (CC4CL). Part I: A framework applied to multiple satellite imaging sensors. Atmos. Meas. Tech., 11, 3373–3396.

USGS, 1996: Global 30 arc seconds topography database, GTOPO30, (https://www.usgs.gov/centers/eros/science/usgs-eros-archive-digital-elevation-global-30-arc-second-elevation-gtopo30#overview)

Wang, D., Liang, S., He, T., Yu, Y., Schaaf, C., and Wang, Z. (2015), Estimating daily mean land surface albedo from MODIS data. J. Geophys. Res. Atmos., 120, 4825– 4841. doi: 10.1002/2015JD023178.

Related articles

• Page:
  ECV - Glaciers, Ice Sheets, Lakes, Soil Moisture
• Page:
  IV Version 1.2; SEC Version 2.0; GMB Version 2.0: Product User Guide Specification (PUGS)
• Page:
  IV Version 1.2; SEC Version 2.0; GMB Version 2.0: System Quality Assurance Document (SQAD)
• Page:
  IV Version 1.2; SEC Version 2.0; GMB Version 2.0: Product Quality Assurance Document (PQAD)
• Page:
  IV Version 1.2; SEC Version 2.0; GMB Version 2.0: Product Quality Assessment Report (PQAR)