Last modified on Nov 08, 2023 18:09

Contributors: N. Clerbaux (Royal Meteorological Institute of Belgium (RMIB)), A. Velazquez Blazquez (RMIB), E. Baudrez (RMIB), C. Aebi (RMIB), T. Akkermans (RMIB)

Issued by: N. Clerbaux (RMIB)

Date: 11/10/2023

Ref: C3S2_D312a_Lot1.1.2.1-v1.0_202310_PQAD_ECVEarthRadiationBudget_v1.3

Official reference number service contract: 2021/C3S2_312a_Lot1_DWD/SC1

History of modifications

List of datasets covered by this document

Related documents

Acronyms

List of tables

List of figures

General definitions

Table 1: Definition of special terms

Scope of the document

This Product Quality Assurance Document (PQAD) provides a description of the product validation for the HIRS daily Outgoing Longwave Radiation (OLR) Climate Data Record (CDR), in its version v01r02. This CDR is brokered into the Climate Data Store (CDS) from National Oceanic and Atmospheric Administration (NOAA) / National Centers for Environmental Information (NCEI). This document summarizes the validations [D2] performed under the NOAA’s Climate Data Record Program. The document also presents some additional validations, in particular to address the quality of the recent data that have been appended to the CDR after the release of [D2].

Executive summary

This document is the Product Quality Assurance Document for the brokered HIRS daily OLR CDR produced by NOAA/NCEI. This version (v01r02) of the daily OLR CDR had been released in 2014, and new data have been added since then.

The dataset comprises more than 43 years (1979-present) of satellite-based measurements derived from the High-resolution Infrared Radiation Sounders (HIRS) instruments onboard the NOAA and Metop satellites. The TCDR (i.e. the “final” version) and ICDR (i.e. the “preliminary” version) provide level-3 data (daily means) on a regular global latitude-longitude grid (with a resolution of 1° x 1°).

In section 1, the validated products are briefly described and the references for more detailed information are presented. In section 2, the reference datasets used for the validation process are described and related to the validated products. Section 3 presents the methodology for the validation of the HIRS daily dataset. The involved metrics are defined and the triple collocation method is explained. Finally, some validation results are presented in section 4.

1. Validated products

1.1 Introduction

The NOAA/NCEI, the producer of the HIRS OLR data, provides the following executive summary description for the Outgoing Longwave Radiation and the version v01r02 of the CDR:

The daily Outgoing Longwave Radiation (OLR) Climate Data Record (CDR) measures the amount of terrestrial radiation released into space and, by extension, the amount of cloud cover and water vapor that intercepts that radiation in the atmosphere. Input data for the daily OLR record primarily comes from the high-resolution infrared radiation sounder (HIRS). The final record is generated through a combination of statistical techniques, including OLR regression, instrument ambient temperature prediction coefficients, and inter-satellite bias corrections.

This Climate Data Record (CDR) contains the daily mean Outgoing Longwave Radiation (OLR) time series in global 1 degree x 1 degree equal-angle gridded maps spanning from January 1, 1979 to December 31, 2013, and continuing daily with a two-day lag. The OLR is estimated directly from the HIRS radiance observations for all sky conditions. The observations from imagers onboard international operational geostationary satellites are incorporated to improve the sampling of the OLR diurnal variation. The Daily OLR CDR is at its initial version 1.2. The data file format is netCDF-4 with CF metadata, and it is accompanied by algorithm documentation, data flow diagram and source code for the NOAA CDR Program.

The version v01r02 of the HIRS OLR CDR provides daily mean Outgoing Longwave Radiation (OLR) in all sky conditions. Now, the dataset comprises more than 43 years (January 1979 onward) of satellite-based measurements derived from the High-Resolution Infrared Radiation Sounder (HIRS) instruments onboard the polar orbiting NOAA and MetOp A/B satellites. The HIRS instruments will be discontinued and the OLR record will be pursued using data from infrared spectrometers (Infrared Atmospheric Sounding Interferometer (IASI) and Cross-track Infrared Sounder (CrIS)).

The HIRS OLR CDR is a level 3 product (daily means) on a regular global latitude-longitude grid with 1° x 1° resolution A summary of the general characteristics of the daily mean HIRS OLR CDR is presented in the Table 1-1. Furthermore, from this dataset a monthly mean product with the same spatial resolution is generated and used in the validation process described in this document.

The detailed description of the algorithm used to generate the daily mean HIRS OLR is given in the Climate Algorithm Theoretical Basis Document (C-ATBD) for Daily OLR CDR v01r02 [D1].The technical aspects related to the brokering of this CDR in the C3S Climate Data Store (CDS) are addressed in a System Quality Assurance Document (SQAD) [D3].

Table 1-1: General characteristics of the HIRS OLR v01r02 daily mean dataset

The product has been intensively evaluated under the NOAA’s Climate Data Record Program The results are summarized in the “Quality Assurance Results and Summary – Outgoing Longwave Radiation (OLR) - Monthly and Daily (Rev. 2.2, dated 08/31/2018)” [D2]⁴.

⁴ Available at: http://olr.umd.edu/References/QA_Summary_OLR-Monthly_and_Daily_CDR_20180831.pdf

The developers of the CDR maintain a portal for data access and relevant documentation. This portal⁵ gives access to the documents (see Table 1-2) related to the HIRS OLR CDR. C-ATBD describes algorithm and software package. Quality Assurance Summary and Results provides updated assessment of the Monthly and Daily OLR CDR products and summarize the Quality Assurance results, supplementing the existing C-ATBD document. The peer-reviewed journal articles describe all steps the development of the dataset, presents intercomparsion studies and discuss a practical application of the datasets, while the conference slides focus on specific aspects of the research process, such as radiometric normalization and homogenization of satellite observations for climate applications. These documents collectively provide comprehensive information on all aspects of the dataset.

⁵ Available at: http://olr.umd.edu

Table 1-2: Relevant documentation concerning the HIRS OLR CDR

1.2 Final and Preliminary versions

With the aim to cover the large diurnal cycle of OLR, the daily CDR used all available satellite overpasses and exploits geostationary observations to model the diurnal variation between the HIRS observations. There are two different versions of the dataset due to a latency (about 6 to 9 months) in the release of the Gridded Satellite data (GridSat) CDR, used for the calibration of the final product. During the latency period a preliminary version is generated using the Geostationary Surface and Insolation Products (GSIP) instead of the GridSat. This preliminary dataset is considered as an Interim Climate Data Record (ICDR) and the final dataset is considered as the Thematic Climate Data Record (TCDR). For the preliminary dataset the word preliminary is included in the name of each product as presented below:

• Final Version: olr-daily_v01r02_20200101_20201231.nc

• Preliminary Version: olr-daily_v01r02-preliminary_20210101_20211231.nc

The preliminary data will be reprocessed as “final” once all the GridSat data are available.

2. Description of validating datasets

2.1 CERES products

The most-suited reference datasets for the validation of the daily and monthly mean HIRS OLR CDR are the Cloud and Earth Radiant Energy System (CERES, Wielicki et al., 1996). The CERES team provides different level 3 products: the Energy Balanced And Filled (EBAF, Loeb et al, 2018), the Synoptic TOA and surface fluxes and clouds (SYN-1deg) (Doelling et al., 2013, 2017) and the Single Scanner Footprint (SSF). These products are available as monthly mean and daily mean (except EBAF which is only monthly mean) at 1°x1° spatial resolution, so they can be directly compared with the HIRS OLR products. The intercomparison with CERES is restricted to the March 2000-onward period (with slightly lower quality before the inclusion of CERES Aqua in 2002). All the CERES data have been downloaded from the CERES Data Products site⁶.

⁶ https://ceres.larc.nasa.gov/data/

2.2 CM SAF GERB/SEVIRI data records

In [D4], an intercomparison is performed between the HIRS OLR, the CERES products and the CM SAF GERB/SEVIRI data records (Clerbaux et al., 2017). Assuming that the errors affecting those three records are uncorrelated, it is possible to estimate the individual uncertainties for these three CDRs. This “triple collocation” approach has been applied and is documented in Section 3.2 (Triple collocation methodology) and Section 1.1 (Validation results using triple collocation method between HIRS OLR, CERES SYN and CM SAF GERB products).

2.3 Long-term records use for stability assessment

To assess the accuracy and stability of the HIRS OLR record before the CERES era, from 1979 to 2000, intercomparisons with respect to different long records are performed and reported on in Section 4.3. These long records are summarized in Table 2-1.

Table 2-1: List of datasets used for validation

3. Description of product validation methodology

In this section, the methodology for the validation of the HIRS daily dataset is explained. The involved metrics are defined and the triple collocation method is explained.

3.1 Bias, root mean square difference (RMSD) and standard deviation

Standard statistical metrics are used for the validation of the dataset. When comparing two OLR maps, OLR₁(x,y) and OLR₂(x,y), we can define the following quantities

in which the sums are done on the 180x360 grid boxes (x,y), and w(x) is a weighting proportional to the Earth surface of the 1°x1° box. The bias represents the difference in global mean OLR between the 2 datasets. The RMSD is the Root Mean Square of the difference and the standard deviation (stddev) is the RMSD corrected for the overall bias. These 3 quantities are expressed in W/m² units.

3.2 Triple collocation methodology

In [D3], intercomparisons are performed between HIRS OLR, CERES and the CM SAF GERB/SEVIRI data records. When three data sources are available, it is possible to convert the observed standard deviations between the records into individual accuracies. This is only possible under the assumption that the errors are not correlated. In [D3] the assumption about un-correlation of the errors is supported by the fact that the records are derived from totally different space instruments (respectively GERB/SEVIRI, CERES, HIRS) operated from different satellites and orbits (geostationary for MSG, polar for Aqua+Terra, polar for NOAA+MetOp).

By denoting A,B,C as the 3 data records and if the errors have normal distributions and are uncorrelated, we can write

where

is the root mean square of either the difference (left terms) or of the error of the dataset. The previous relations can be inverted into:

The main results are presented in this document in Section 4.2.

4. Validation results

4.1 Results of comparison with CERES from [D2]

This section summarizes the main results of the intercomparison of HIRS OLR CDR v01r02 with CERES EBAF Edition 4 as reported in [D2]. The comparison is done over the period from March 2000 to February 2018. The daily HIRS OLR products have been monthly averaged before comparison with EBAF. When comparing with CERES, it is assumed that the errors are not correlated which seems like a reasonable assumption as CERES and HIRS are totally different instruments and are flying on different orbits. The intercomparison with CERES is likely to slightly overestimate the error affecting the HIRS OLR CDR because it incorporates some uncertainty due to the CERES estimation (e.g. spatial and temporal sampling, angular dependency model).

Figure 4-1 shows the spatial variations of the mean difference between the products. In general, the HIRS OLR is slightly lower than CERES EBAF. Figure 4-2 shows the time series of the global mean difference. Although there is a clear seasonal cycle, a clear average bias exists of about 2.3 W/m² and standard deviation average value of 1.5 W/m² [D2]. This bias seems to remain stable during the 2002-2016 time period, complying with the 0.3W/m²/decade stability requirement defined by GCOS (according to [D2]). Figure 4-3 shows the spatial variations of the standard deviation (RMS of the difference corrected by the bias) between the products. The standard deviation remains stable during the 2002-2016 time period, but shows slightly higher values before July 2002. This is likely due to degraded CERES products before July 2002, when they can use only CERES data from the Terra satellite.

Figure 4-1: Mean difference between daily mean HIRS OLR v01r02 and CERES EBAF Ed 4.0, over the period March 2000 - February 2018 (extracted from [D2], Fig. 21 left).

Figure 4-2: Timeseries of global mean monthly OLR difference (red), the standard deviation of the differences (cyan), and the RMS difference (blue) and between monthly-integrated v01r02 Daily OLR CDR and CERES EBAF Ed 4.0 (extracted from [D2], Fig. 20).

Figure 4-3: Standard deviation of the difference between daily mean HIRS OLR v01r02 and CERES EBAF Ed 4.0 over the period March 2000 – December 2016 (extracted from [D2], Fig. 21 right).

4.2 Validation results using triple collocation method between HIRS OLR, CERES SYN and CM SAF GERB products

We have applied the triple collocation method (see Section 3.2) to the all sky monthly mean OLR data from CM SAF, CERES SYN and HIRS. The assumption about no correlation between the errors is supported by the fact that the records are derived from different instruments (respectively GERB/SEVIRI, CERES, HIRS) operated from different satellites and orbits (geostationary for MSG, polar for Aqua+Terra, polar for NOAA+MetOp).

The result is shown in Figure 4. The averaged accuracies are estimated at: 1.6 W/m² for CM SAF and 0.9 W/m² for SYN MM (Monthly Mean) and 0.9 W/m² for HIRS v01r02 monthly mean. Note that these numbers are estimated over the 60°N-60°S and 60°W-60°E region and can therefore differ from the global mean numbers given in Section 4.1.

Figure 4-4: Individual accuracies of the monthly mean OLR (here called TOA Emitted Thermal TET) from CM SAF GERB/SEVIRI ed02, CERES SYN1deg-month ed4.0 and HIRS v01r02, as estimated from their RMS differences (extracted from [D4]).

4.3 Comparison with long data records

Figure 4-5 shows the global mean bias between HIRS v01r02 MM and several of the other CDRs described in Section 2. In general, the HIRS OLR record looks stable but the following is worth noting for possible further investigation/reprocessing:

• From 2018 onward, the HIRS OLR seems to decrease with respect to the CERES, CLARA and ERA5. This is further discussed in the next section.

• A couple of months around December 2000 seem to be affected by higher negative bias.

Except for this, there is no evidence of jumps or drifts that would affect the HIRS OLR v01r02 CDR.

Figure 4-5: Bias of the HIRS OLR v01r02 with respect to CERES EBAF, CERES SYN, ERA5, ISCCP, CLOUD CCI and CLARA-A3.

4.4 Recent data evaluation (as of February 2022)

This section addresses the evaluation of the recent data, including the interim (ICDR) data. Figure 4-6 shows the global monthly mean difference between HIRS OLR v01r02 and CERES EBAF ed4.1 (that was available until November 2021 at time of writing). The 2.5% and 97.5% percentiles of the bias have been determined on the “final” part of the CDR. Figure 4-7: Standard deviation between HIRS OLR v01r02 and CERES EBAF d4.1. From 2018 onward, a decrease of OLR is well visible and is attributed to the orbital drift of the afternoon satellites (NOAA18 and NOAA19). The maximum negative bias seemed to have occurred in January 2021 and the level has been recovering during 2021. This stability problem is expected to be removed in a further release of the HIRS OLR CDR that will ingest also CrIS and IASI data for the satellites without HIRS instrument (NPP, NOAA20 and successors, Metop-C).

Figure 4-7 shows the standard deviation between the products. The higher uncertainty between March 2000 and July 2003 is explained by the absence of CERES afternoon observations. From 2018 onward, the slight increase of the standard deviation is attributed to the NOAA18 and NOAA19 orbit drift.

Figure 4-6: Global monthly mean OLR difference between HIRS OLR v01r02 and CERES EBAF ed4.1. The 2.5% and 97.5% percentiles are evaluated on the “final” data (up to 31/12/2020).

Figure 4-7: Standard deviation between HIRS OLR v01r02 and CERES EBAF d4.1.

4.5 Summary

4.5.1 Accuracy

Using CERES EBAF as the reference, the accuracy (standard deviation) of the HIRS OLR product is evaluated at 1.5 W/m² [D2]. A part of this value could however be attributed to errors in the CERES products, so the uncertainty in the monthly mean HIRS OLR values is likely better than 1.5 W/m². Assuming a similar uncertainty level for the HIRS OLR and CERES products, the observed 1.5 W/m² standard deviation would reduce to 0.9 W/m² uncertainties for both products. Such a level of uncertainty is supported by the triple collocation of CERES, HIRS and GERB OLR.

4.5.2 Stability

The HIRS OLR record shows good stability with respect to CERES but a drift towards more negative bias is observed after 2018. Stability over the pre-CERES era is more difficult to assess due to lack of a stable OLR reference. However, the timeseries of anomalies with respect to ERA5, CLARA-A3, Cloud CCI, ISCCP-FH products do not show any obvious stability problem.

References

Clerbaux, N., Urbain, M., Ipe, A., Baudrez, E., Velazquez-Blazquez, A., Akkermans, T., Hollmann, R., Fuchs, P., Selbach, N., Werscheck, M., 2017: CM SAF TOA Radiation GERB/SEVIRI Data Record - Edition 2, Satellite Application Facility on Climate Monitoring, DOI:10.5676/EUM_SAF_CM/​TOA_GERB/V002.

Doelling, D. R., M. Sun, L. T. Nguyen, M. L. Nordeen, C. O. Haney, D. F. Keyes, P. E. Mlynczak, 2016: Advances in Geostationary-Derived Longwave Fluxes for the CERES Synoptic (SYN1deg) Product, Journal of Atmospheric and Oceanic Technology, 33(3), 503-521. doi: 10.1175/JTECH-D-15-0147.1

Doelling, D. R., N. G. Loeb, D. F. Keyes, M. L. Nordeen, D. Morstad, C. Nguyen, B. A. Wielicki, D. F. Young, M. Sun, 2013: Geostationary Enhanced Temporal Interpolation for CERES Flux Products, Journal of Atmospheric and Oceanic Technology, 30(6), 1072-1090. doi: 10.1175/JTECH-D-12-00136.1.

Hersbach, H., Bell, B., Berrisford, P., Hirahara, S., Horányi, A., Muñoz‐Sabater, J., Nicolas, J., Peubey, C., Radu, R., Schepers, D. and Simmons, A., 2020: The ERA5 global reanalysis. Quarterly Journal of the Royal Meteorological Society, 146(730), pp.1999-2049.

Hogan, R.J. and Bozzo, A., 2018. A flexible and efficient radiation scheme for the ECMWF model. Journal of Advances in Modeling Earth Systems, 10(8), pp.1990-2008.

Knapp, K. R., S. Ansari, C. L. Bain, M. A. Bourassa, M. J. Dickinson, C. Funk, C. N. Helms, C. C. Hennon, C. D. Holmes, G. J. Huffman, J. P. Kossin, H.-T. Lee, A. Loew, and G. Magnusdottir, 2011: Globally gridded satellite (GridSat) observations for climate studies. Bulletin of the American Meteorological Society, 92, 893-907. doi:10.1175/2011BAMS3039.1

Lee, H.-T.; Heidinger, A.; Gruber, A.; Ellingson, R.G.: The HIRS outgoing longwave radiation product from hybrid polar and geosynchronous satellite observations. Adv. Space Res. 2004, 33, 1120–1124

Lee, H.-T., 2014: Daily OLR CDR – Development and Evaluation. CERES Science Team Meeting, Apr 2014

Lee, H.-T., 2014: Daily OLR Climate Data Record. EGU General Assembly, Apr 2014

Lee, H.-T., 2018: Quality Assurance Summary and Results for Monthly and Daily OLR CDR (rev.20180831).

Lee, H.-T., A. Gruber, R. G. Ellingson and I. Laszlo, 2007: Development of the HIRS Outgoing Longwave Radiation climate data set. J. Atmos. Ocean. Tech., 24, 2029–2047.

Lee, H.-T., C. J. Schreck, and K. R. Knapp, 2014: Generation of Daily OLR CDR. Eumetsat Meteorological Satellite Conference, Sep 2014

Loeb, N. G., D. R. Doelling, H. Wang, W. Su, C. Nguyen, J. G. Corbett, L. Liang, C. Mitrescu, F. G. Rose, and S. Kato, 2018: Clouds and the Earth’s Radiant Energy System (CERES) Energy Balanced and Filled (EBAF) Top-of-Atmosphere (TOA) Edition-4.0 Data Product. J. Climate, 31, 895-918, doi: 10.1175/JCLI-D-17-0208.1

Loeb, N. G., S. Kato, and B. A. Wielicki, 2002: Defining top-of-atmosphere flux reference level for Earth Radiation Budget studies. J. Climate, 15, 3301-3309.

Schreck, C. J., H.-T. Lee and K. Knapp, 2018: HIRS Outgoing Longwave Radiation—Daily Climate Data Record: Application toward Identifying Tropical Subseasonal Variability. Remote Sens. 2018, 10, 1325

Schulz, J., Albert, P., Behr, H.D., Caprion, D., Deneke, H., Dewitte, S., Durr, B., Fuchs, P., Gratzki, A., Hechler, P. and Hollmann, R., 2009: Operational climate monitoring from space: the EUMETSAT Satellite Application Facility on Climate Monitoring (CM-SAF). Atmospheric Chemistry and Physics, 9(5), pp.1687-1709.

Stengel, M., Stapelberg, S., Sus, O., Finkensieper, S., Würzler, B., Philipp, D., Hollmann, R., Poulsen, C., Christensen, M. and McGarragh, G., 2020: Cloud_cci Advanced Very High Resolution Radiometer post meridiem (AVHRR-PM) dataset version 3: 35-year climatology of global cloud and radiation properties. Earth System Science Data, 12(1), pp.41-60.

Wielicki, B.A., Barkstrom, B.R., Harrison, E.F., Lee III, R.B., Smith, G.L. and Cooper, J.E., 1996. Clouds and the Earth's Radiant Energy System (CERES): An earth observing system experiment. Bulletin of the American Meteorological Society, 77(5), pp.853-868.

Young, A. H., Knapp, K. R., Inamdar, A., Hankins, W., and Rossow, W. B., 2018: The International Satellite Cloud Climatology Project H-Series climate data record product, Earth Syst. Sci. Data, 10, 583–593, doi:10.5194/essd-10-583-2018.

Related articles

• Page:
  ECV - Glaciers, Ice Sheets, Lakes, Soil Moisture
• Page:
  IV Version 1.2; SEC Version 2.0; GMB Version 2.0: Product User Guide Specification (PUGS)
• Page:
  IV Version 1.2; SEC Version 2.0; GMB Version 2.0: System Quality Assurance Document (SQAD)
• Page:
  IV Version 1.2; SEC Version 2.0; GMB Version 2.0: Product Quality Assurance Document (PQAD)
• Page:
  IV Version 1.2; SEC Version 2.0; GMB Version 2.0: Product Quality Assessment Report (PQAR)