Page info infoType Modified date prefix Last modified on type Flat
Contributors:A N. Velazquez Blazquez Clerbaux (Royal Meteorological Institute of Belgium (RMIB)), NA. Clerbaux Velazquez Blazquez (RMIB), E. Baudrez (RMIB), C. Aebi (RMIB), T. Akkermans (RMIB)
Issued by: N. Clerbaux (RMIB)
Date: 0811/0710/20212023
Ref: C3S C3S2_D312bD312a_Lot1.1.2.5.51-v1.20_202103202310_PQAD_ECVEarthRadiationBudget_v1.23
Official reference number service contract: 2018 2021/C3SC3S2_312b312a_Lot1_DWD/SC1
Info | ||||
---|---|---|---|---|
| ||||
|
Easy Heading Macro | ||
---|---|---|
|
History of modifications
Expand | |||||||||||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| |||||||||||||||||||||||||||||||
|
List of datasets covered by this document
Expand | ||||||||||||||||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ||||||||||||||||||||||||||||||||||||
| note1 | note1 | 1The deliverable ID is an internal reference for contract C3S 312bC3S2 312a Lot1
Anchor | ||||
---|---|---|---|---|
|
3 The deliverable title under the C3S2_312a contract is Earth Radiation Budget OLR_HIRS ICDR v2.x Anchor note3 note3
Anchor related_documents related_documents
Related documents
related_documents | |
related_documents |
Expand | |||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| |||||||||||||||
|
Acronyms
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Expand | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
List of tables
Expand | ||
---|---|---|
| ||
Table 1-1: Relevant documentation concerning the HIRS OLR CDR Table 4-1: Summary of KPI results with 2.5 and 97.5 percentiles and number of ICDR months within the range |
List of figures
Expand | ||
---|---|---|
| ||
Figure 4-1: Mean difference between monthly mean HIRS OLR v02r07 and CERES EBAF Edition 4, over March 2000 - February 2018 Figure 4-2: Timeseries of global mean difference between monthly mean HIRS OLR v02r07 (red curve) and CERES EBAF Edition 4 over Mar 2000 - Feb 2018 Figure 4-3: Standard deviation (aka “bias-corrected RMS”) of the difference between monthly mean HIRS OLR v02r07 and CERES EBAF Edition 4 over Mar 2000 - Feb 2018 Figure 4-4: Timeseries of the standard deviation of the difference between monthly mean HIRS OLR v02r07 (red curve) and CERES EBAF Edition 4 over Mar 2000 - Feb 2018 Figure 4-5: Individual accuracies of the monthly mean OLR (here called TOA Emitted Thermal TET) from CM SAF, CERES SYN and HIRS, as estimated from their RMS differences |
Scope of the document
This Product Quality Assurance Document provides a description of the whole product validation methodology for the HIRS Outgoing Longwave Radiation (OLR) Climate Data Record (CDR). This CDR is "brokered" into the Climate Data Store (CDS) from National Oceanic and Atmospheric Administration (NOAA) / National Centers for Environmental Information (NCEI). Therefore, this document only provides links to the relevant validation documents and published papers.
Executive summary
The NOAA/NCEI, the producer of the HIRS OLR data, provides the following executive summary for the v2.7 of the CDR:
This Climate Data Record (CDR) of monthly mean Outgoing Longwave Radiation (OLR) flux at the top of the atmosphere in all sky conditions is on a 2.5 degree x 2.5 degree grid with global coverage from January 1979 to the present and continuing monthly. It is derived based on the multispectral High Resolution Infrared Radiation Sounder (HIRS) OLR algorithm and other methodologies. This dataset was created from HIRS OLR retrievals from TIROS-N to NOAA-19, and MetOp-A/B satellites with inter-satellite calibration adjustments and employed the empirical diurnal model for monthly mean derivation. This version 2.7 is an upgrade from the previous version 2.2-1. The main change is in the OLR regression models and the inter-satellite calibration. The version 2.7 OLR regression models improved the retrieval consistency between variant versions of HIRS instruments, HIRS-2, 2i, 3 and 4, and in turn, the accuracy of inter-satellite calibration is significantly improved, thus eliminating the spurious trends presented in the version 2.2-1 data. The data file format is netCDF-4 with CF metadata, and it is accompanied by algorithm documentation, data flow diagram and source code for the NOAA CDR Program.
This version (v02r07 or simply "2.7") of the OLR-monthly CDR has been released on 31/08/2018, and new data have been added since then. Now, it contains more than 40 years of satellite-based measurements of the OLR, on a global scale and at 2.5° x 2.5° spatial resolution. Table 1 provides a summary of the accuracies for the HIRS OLR CDR.
|
Acronyms
Expand | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
List of tables
Expand | ||
---|---|---|
| ||
Table 1-1: General characteristics of the HIRS OLR v01r02 daily mean dataset Table 1-2: Relevant documentation concerning the HIRS OLR CDR Table 2-1: List of datasets used for validation |
List of figures
Expand | ||
---|---|---|
| ||
Figure 4-1: Mean difference between daily mean HIRS OLR v01r02 and CERES EBAF Ed 4.0, over the period March 2000 - February 2018 (extracted from [D2], Fig. 21 left). Figure 4-2: Timeseries of global mean monthly OLR difference (red), the standard deviation of the differences (cyan), and the RMS difference (blue) and between monthly-integrated v01r02 Daily OLR CDR and CERES EBAF Ed 4.0 (extracted from [D2], Fig. 20). Figure 4-3: Standard deviation of the difference between daily mean HIRS OLR v01r02 and CERES EBAF Ed 4.0 over the period March 2000 – December 2016 (extracted from [D2], Fig. 21 right). Figure 4-4: Individual accuracies of the monthly mean OLR (here called TOA Emitted Thermal TET) from CM SAF GERB/SEVIRI ed02, CERES SYN1deg-month ed4.0 and HIRS v01r02, as estimated from their RMS differences (extracted from [D4]). Figure 4-5: Bias of the HIRS OLR v01r02 with respect to CERES EBAF, CERES SYN, ERA5, ISCCP, CLOUD CCI and CLARA-A3. Figure 4-6: Global monthly mean OLR difference between HIRS OLR v01r02 and CERES EBAF ed4.1. The 2.5% and 97.5% percentiles are evaluated on the “final” data (up to 31/12/2020). Figure 4-7: Standard deviation between HIRS OLR v01r02 and CERES EBAF d4.1. |
General definitions
Table 1: Summary of the accuracy of the monthly mean OLR at 2.5° x 2.5° resolution provided in v02r07 of the CDR. The uncertainty is provided at "1 standard deviation" from the intercomparison with CERES EBAF Edition 4.0 (see section 4.1). Anchor table1 table1
...
Product Name
...
Processing uncertainties
...
OLR - Outgoing Longwave Radiation
(field in the HIRS OLR NetCDF files : olr )
...
1.8 W/m²
1. Validated products
The version v02.r07 of the HIRS OLR CDR provides monthly mean Outgoing Longwave Radiation (OLR) in all sky conditions. The dataset comprises 42 years and 5 months (January 1979 – Feb 2021) of satellite-based measurements derived from the High-Resolution Infrared Radiation Sounder (HIRS) instruments onboard the polar orbiting NOAA and MetOp satellites.
The HIRS OLR CDR is a level 3 product (monthly means) on a regular global latitude-longitude grid with 2.5° x 2.5° resolution. The detailed description of the algorithm used to generate the monthly mean HIRS OLR is given in the Climate Algorithm Theoretical Basis Document (C-ATBD) for Monthly OLR CDR v02r07 [D2]. A Product User Guide and Specification document [D1] is also issued for the brokering of this CDR in the CDS.
The product has been intensively evaluated by Dr. Hai-Tien Lee (University of Maryland), the PI of the HIRS OLR product. The results are summarized in the “Quality Assurance Results and Summary” document [D5].
The developers of the CDR maintain a portal for data access and relevant documentation at:
This portal gives access to the following documents (see Table 1-1) related to the HIRS OLR CDR.
...
Documents and URLs
...
Lee, H.-T., 2018: Quality Assurance Summary and Results for Monthly and Daily OLR CDR (rev.20180831).
http://olr.umd.edu/References/QA_Summary_OLR-Monthly_and_Daily_CDR_20180831.pdf
...
2. Description of validating datasets
The best reference dataset for the validation of the monthly mean HIRS OLR CDR are the CERES datasets (CERES EBAF (Loeb et al., 2018) and CERES SYN1deg-month (Doelling et al., 2013 and 2016). However, the validation is then restricted to the 2000-onward period (with slightly lower quality before the inclusion of CERES Aqua in 2002).
In [D3], an intercomparison is performed between HIRS OLR, CERES EBAF and the CM SAF GERB/SEVIRI data records (Clerbaux et al., 2017). Assuming that the errors affecting those three records are uncorrelated, it is possible to estimate the individual error for the three CDRs.
...
The comparison with CERES EBAF is described in section 7.1 of the “Quality Assurance Results and Summary” [D5]. This provides bias and standard deviation. Each of the statistics is reported “regionally” (i.e. in 2.5°x2.5° boxes) and as a time series of global values.
The main findings of the HIRS/CERES intercomparison [D5] are reproduced in section 4.1.
In [D3], intercomparisons are performed between HIRS OLR, CERES EBAF (Edition 2.8) and the CM SAF GERB/SEVIRI ed02 data records. When at least three data records are available, it is possible to convert the observed standard deviations between the records into individual accuracies. This is only possible if the errors are not correlated. The assumption about un-correlation of the errors is supported by the fact that the data records are derived from totally different space instruments (respectively GERB/SEVIRI, CERES, HIRS) operated from different satellites/orbits (geostationary for MSG, polar for Aqua+Terra, polar for NOAA+MetOp).
By denoting the 3 data records A, B, C, and if the errors have normal distributions and are uncorrelated, we can write
Mathdisplay |
---|
\sigma^2 (A-B) = \sigma^2(A) + \sigma^2(B) |
Mathdisplay |
---|
\sigma^2 (A-C) = \sigma^2(A) + \sigma^2(C) |
Mathdisplay |
---|
\sigma^2 (B-C) = \sigma^2(B) + \sigma^2(C) |
where s( ) is the standard deviation of either the difference (left terms) or of the error of the dataset. The previous relations can be inverted into:
Mathdisplay |
---|
\sigma^2 (A) = 0.5 * (\sigma^2(A-B) + \sigma^2(A-C) - \sigma^2(B-C)) |
Mathdisplay |
---|
\sigma^2 (B) = 0.5 * (\sigma^2(A-B) + \sigma^2(B-C) - \sigma^2(A-C)) |
Mathdisplay |
---|
\sigma^2 (C) = 0.5 * (\sigma^2(A-B) + \sigma^2(B-C) - \sigma^2(A-B)) |
The main results are reproduced hereafter, in section 4.2.
4. Summary of validation results
...
This section summarizes the main results of the intercomparison of HIRS OLR CDR v02r07 with CERES EBAF Edition 4 as reported in [D5]. The comparison is done over the period from March 2000 to February 2018.
Figure 4-1 shows the spatial variations of the mean difference between the products. In general, the HIRS OLR is slightly lower than CERES EBAF. Figure 4-2 shows the time series of the global mean difference. Although there is a clear seasonal cycle, a clear average bias exists of about 1.6 W/m² [D5]. This bias seems to remain stable during the 2002-2018 time period, probably complying with the 0.3W/m²/decade stability requirement defined by GCOS.
Figure 4-3 shows the spatial variations of the standard deviation (RMS of the difference corrected by the bias) between the products. Figure 4-4 shows the time series of the standard deviation which has an average value of 1.8 W/m² [D5]. The standard deviation remains stable during the 2002-2018 time period but shows slightly higher values before July 2002. This is likely due to degraded CERES products before July 2002, when they can use only CERES data from the Terra satellite.
...
...
...
Definition of special terms
Jargon | Definition |
---|---|
Brokered products | The C3S Climate Data Store (CDS) provides both data produced specifically for C3S and so-called brokered products. The latter are existing products produced under an independent programme or project which are made available through the CDS. |
Climate Data Store (CDS) | The front-end and delivery mechanism for data made available through C3S. |
OLR | The Outgoing Longwave Radiation is the total quantity of radiant energy emitted by the Earth-atmosphere system that escapes at the Top-of-Atmosphere in Watts per square meter. |
Level 1 products | Reconstructed, unprocessed instrument data, time-referenced. |
Level 2 products | Derived geophysical variables at the same resolution and location as L1 source data. |
Level 3 products | Variables are mapped on uniform space-time grid scales, usually with some completeness and consistency. |
Infrared spectrometers | Instrument that collects infrared spectra measurements based on measurements of the coherence of a radiative source, using time-domain or space-domain measurements of the radiation. |
Triple collocation methodology | Triple collocation is a method used to characterize the uncertainties when uncertainty in triplets of measurements of a same quantity are available. |
Bias | The systematic difference between the estimator's expected value and the true value of the parameter being estimated. |
RMSD | The Root Mean Square Deviation is the square root of the mean squared error. |
Standard deviation | RMSD corrected for the overall bias. |
Percentile | It is a measure used in statistics indicating the value below which a given percentage of observations in a group of observations fall. |
Scope of the document
This Product Quality Assurance Document (PQAD) provides a description of the product validation for the HIRS daily Outgoing Longwave Radiation (OLR) Climate Data Record (CDR), in its version v01r02. This CDR is brokered into the Climate Data Store (CDS) from National Oceanic and Atmospheric Administration (NOAA) / National Centers for Environmental Information (NCEI). This document summarizes the validations [D2] performed under the NOAA’s Climate Data Record Program. The document also presents some additional validations, in particular to address the quality of the recent data that have been appended to the CDR after the release of [D2].
Executive summary
This document is the Product Quality Assurance Document for the brokered HIRS daily OLR CDR produced by NOAA/NCEI. This version (v01r02) of the daily OLR CDR had been released in 2014, and new data have been added since then.
The dataset comprises more than 43 years (1979-present) of satellite-based measurements derived from the High-resolution Infrared Radiation Sounders (HIRS) instruments onboard the NOAA and Metop satellites. The TCDR (i.e. the “final” version) and ICDR (i.e. the “preliminary” version) provide level-3 data (daily means) on a regular global latitude-longitude grid (with a resolution of 1° x 1°).
In section 1, the validated products are briefly described and the references for more detailed information are presented. In section 2, the reference datasets used for the validation process are described and related to the validated products. Section 3 presents the methodology for the validation of the HIRS daily dataset. The involved metrics are defined and the triple collocation method is explained. Finally, some validation results are presented in section 4.
Anchor | ||||
---|---|---|---|---|
|
Anchor | ||||
---|---|---|---|---|
|
The NOAA/NCEI, the producer of the HIRS OLR data, provides the following executive summary description for the Outgoing Longwave Radiation and the version v01r02 of the CDR:
The daily Outgoing Longwave Radiation (OLR) Climate Data Record (CDR) measures the amount of terrestrial radiation released into space and, by extension, the amount of cloud cover and water vapor that intercepts that radiation in the atmosphere. Input data for the daily OLR record primarily comes from the high-resolution infrared radiation sounder (HIRS). The final record is generated through a combination of statistical techniques, including OLR regression, instrument ambient temperature prediction coefficients, and inter-satellite bias corrections.
This Climate Data Record (CDR) contains the daily mean Outgoing Longwave Radiation (OLR) time series in global 1 degree x 1 degree equal-angle gridded maps spanning from January 1, 1979 to December 31, 2013, and continuing daily with a two-day lag. The OLR is estimated directly from the HIRS radiance observations for all sky conditions. The observations from imagers onboard international operational geostationary satellites are incorporated to improve the sampling of the OLR diurnal variation. The Daily OLR CDR is at its initial version 1.2. The data file format is netCDF-4 with CF metadata, and it is accompanied by algorithm documentation, data flow diagram and source code for the NOAA CDR Program.
The version v01r02 of the HIRS OLR CDR provides daily mean Outgoing Longwave Radiation (OLR) in all sky conditions. Now, the dataset comprises more than 43 years (January 1979 onward) of satellite-based measurements derived from the High-Resolution Infrared Radiation Sounder (HIRS) instruments onboard the polar orbiting NOAA and MetOp A/B satellites. The HIRS instruments will be discontinued and the OLR record will be pursued using data from infrared spectrometers (Infrared Atmospheric Sounding Interferometer (IASI) and Cross-track Infrared Sounder (CrIS)).
The HIRS OLR CDR is a level 3 product (daily means) on a regular global latitude-longitude grid with 1° x 1° resolution A summary of the general characteristics of the daily mean HIRS OLR CDR is presented in the Table 1-1. Furthermore, from this dataset a monthly mean product with the same spatial resolution is generated and used in the validation process described in this document.
The detailed description of the algorithm used to generate the daily mean HIRS OLR is given in the Climate Algorithm Theoretical Basis Document (C-ATBD) for Daily OLR CDR v01r02 [D1].The technical aspects related to the brokering of this CDR in the C3S Climate Data Store (CDS) are addressed in a System Quality Assurance Document (SQAD) [D3].
Table 1-1: General characteristics of the HIRS OLR v01r02 daily mean dataset Anchor table1_1 table1_1
General characteristics of daily mean HIRS OLR v01r02 | |
---|---|
Spatial resolution | 1° x 1° |
Grid | Regular lat-lon (180x360 grid boxes) |
Temporal resolution | daily mean |
Time period | January 1979 to present |
Format | NetCDF version 4, CF compliant |
Reference level for the fluxes | 20km above mean sea level (Loeb et al, 2002) |
Geophysical quantity | Outgoing Longwave Radiation (OLR), also known as “longwave flux” or “thermal flux” |
The product has been intensively evaluated under the NOAA’s Climate Data Record Program The results are summarized in the “Quality Assurance Results and Summary – Outgoing Longwave Radiation (OLR) - Monthly and Daily (Rev. 2.2, dated 08/31/2018)” [D2]4.
4 Available at: http://olr.umd.edu/References/QA_Summary_OLR-Monthly_and_Daily_CDR_20180831.pdf Anchor note4 note4
The developers of the CDR maintain a portal for data access and relevant documentation. This portal5 gives access to the documents (see Table 1-2) related to the HIRS OLR CDR. C-ATBD describes algorithm and software package. Quality Assurance Summary and Results provides updated assessment of the Monthly and Daily OLR CDR products and summarize the Quality Assurance results, supplementing the existing C-ATBD document. The peer-reviewed journal articles describe all steps the development of the dataset, presents intercomparsion studies and discuss a practical application of the datasets, while the conference slides focus on specific aspects of the research process, such as radiometric normalization and homogenization of satellite observations for climate applications. These documents collectively provide comprehensive information on all aspects of the dataset.
5 Available at: http://olr.umd.edu Anchor note5 note5
Table 1-2: Relevant documentation concerning the HIRS OLR CDR Anchor table1_2 table1_2
Documents and URLs |
---|
Schreck, C. J., H.-T. Lee and K. Knapp, 2018: HIRS Outgoing Longwave Radiation—Daily Climate Data Record: Application toward Identifying Tropical Subseasonal Variability. Remote Sens. 2018, 10, 1325. |
Lee, H.-T., 2018: Quality Assurance Summary and Results for Monthly and Daily OLR CDR (rev.20180831). http://olr.umd.edu/References/QA_Summary_OLR-Monthly_and_Daily_CDR_20180831.pdf |
Lee, H.-T., A. Gruber, R. G. Ellingson and I. Laszlo, 2007: Development of the HIRS Outgoing Longwave Radiation climate data set. J. Atmos. Ocean. Tech., 24, 2029–2047. http://olr.umd.edu/References/Lee_et_al_2007_HIRS_OLR_CDR.pdf |
Lee, H.-T., 2014: Climate Algorithm Theoretical Basis Document (C-ATBD) Outgoing Longwave Radiation (OLR) – Daily. NOAA CDR Program Document Number : CDRP-ATBD-0526. |
Climate Algorithm Theoretical Basis Document (C-ATBD) for Monthly OLR CDR v02r07 |
Lee, H.-T., 2014: Daily OLR CDR – Development and Evaluation. CERES Science Team Meeting, Apr 2014 |
Lee, H.-T., 2014: Daily OLR Climate Data Record. EGU General Assembly, Apr 2014 http://olr.umd.edu/References/Lee_2014_Daily_OLR_Climate_Data_Record_EGU_Apr2014.pdf |
Lee, H.-T., C. J. Schreck, and K. R. Knapp, 2014: Generation of Daily OLR CDR. Eumetsat Meteorological Satellite Conference, Sep 2014 http://olr.umd.edu/References/Lee_2014_Generation_of_Daily_OLR_CDR_Eumetsat_Sep2014.pdf |
Read me for Daily OLR CDR v01r02 http://olr.umd.edu/References/Read%20me%20for%20Daily%20OLR%20CDR%20v01r02.txt |
1.2 Final and Preliminary versions
With the aim to cover the large diurnal cycle of OLR, the daily CDR used all available satellite overpasses and exploits geostationary observations to model the diurnal variation between the HIRS observations. There are two different versions of the dataset due to a latency (about 6 to 9 months) in the release of the Gridded Satellite data (GridSat) CDR, used for the calibration of the final product. During the latency period a preliminary version is generated using the Geostationary Surface and Insolation Products (GSIP) instead of the GridSat. This preliminary dataset is considered as an Interim Climate Data Record (ICDR) and the final dataset is considered as the Thematic Climate Data Record (TCDR). For the preliminary dataset the word preliminary is included in the name of each product as presented below:
Final Version: olr-daily_v01r02_20200101_20201231.nc
Preliminary Version: olr-daily_v01r02-preliminary_20210101_20211231.nc
The preliminary data will be reprocessed as “final” once all the GridSat data are available.
Anchor | ||||
---|---|---|---|---|
|
2.1 CERES products
The most-suited reference datasets for the validation of the daily and monthly mean HIRS OLR CDR are the Cloud and Earth Radiant Energy System (CERES, Wielicki et al., 1996). The CERES team provides different level 3 products: the Energy Balanced And Filled (EBAF, Loeb et al, 2018), the Synoptic TOA and surface fluxes and clouds (SYN-1deg) (Doelling et al., 2013, 2017) and the Single Scanner Footprint (SSF). These products are available as monthly mean and daily mean (except EBAF which is only monthly mean) at 1°x1° spatial resolution, so they can be directly compared with the HIRS OLR products. The intercomparison with CERES is restricted to the March 2000-onward period (with slightly lower quality before the inclusion of CERES Aqua in 2002). All the CERES data have been downloaded from the CERES Data Products site6.
Anchor | ||||
---|---|---|---|---|
|
2.2 CM SAF GERB/SEVIRI data records
In [D4], an intercomparison is performed between the HIRS OLR, the CERES products and the CM SAF GERB/SEVIRI data records (Clerbaux et al., 2017). Assuming that the errors affecting those three records are uncorrelated, it is possible to estimate the individual uncertainties for these three CDRs. This “triple collocation” approach has been applied and is documented in Section 3.2 (Triple collocation methodology) and Section 1.1 (Validation results using triple collocation method between HIRS OLR, CERES SYN and CM SAF GERB products).
2.3 Long-term records use for stability assessment
To assess the accuracy and stability of the HIRS OLR record before the CERES era, from 1979 to 2000, intercomparisons with respect to different long records are performed and reported on in Section 4.3. These long records are summarized in Table 2-1.
Table 2-1: List of datasets used for validation Anchor table2_1 table2_1
Validating dataset | Period | Description | ||||||
---|---|---|---|---|---|---|---|---|
ERA5 ERA5-T | Jan. 1979 to present | TOA fluxes from the latest ECMWF ReAnalysis, ERA5 (Hersbach et al, 2020) have been collected from the Copernicus Climate Data Store at https://cds.climate.copernicus.eu The radiation scheme in ERA5 is described in Hogan and Bozzo (2018). The downloaded data, at 0.25°x0.25° resolution, are aggregated to the 1°x1° grid. | ||||||
CLARA-A3 + ICDR | Jan. 1979 to Dec. 2020 | The Climate Monitoring Satellite Application Facility (CM SAF, Schulz et al., 2009) of EUMETSAT is developing long global CDR of TOA fluxes based on the AVHRR GAC data. These fluxes will be published as part of the CLARA-A3 product portfolio. The RSF and OLR retrieval algorithms are described, respectively, by Akkermans et al. (2020) and Clerbaux et al. (2020). A pre-released version has been used for this work. The data have been aggregated from the 0.25°x0.25° resolution to the 1°x1° grid. The CM SAF data are available from Several fields of the CLARA-A3 record, as well as the ICDR continuation, are expected to be published in the C3S/CDS in 2023. | ||||||
ISCCP-FH | July 1983 to June 2017 | TOA fluxes are also available in the ISCCP H-series of products (Young et al, 2018) https://isccp.giss.nasa.gov/pub/flux-fh/tar-nc4_MPF/7
| ||||||
CLOUD CCI AVHRR | Sep. 1991 to Dec. 2016 | The ESA Cloud CCI (Climate Change Initiative) has developed products providing TOA fluxes computed, using a radiative transfer model (BUGSrad), from the retrieved cloud properties. One of these products is based on the AVHRR GAC data and is described by Stengel et al. (2020). Separate MM products exist for the morning (AM) and for the afternoon (PM) satellites. For this assessment, the AM and PM products have been averaged to produce a unique MM product. The spatial resolution is changed from the 0.5°x0.5° resolution to the 1°x1° grid. The ESA CCI AVHRR have been downloaded from: https://public.satproj.klima.dwd.de/data/ESA_Cloud_CCI/CLD_PRODUCTS/v3.0/L3C/ |
Anchor | ||||
---|---|---|---|---|
|
In this section, the methodology for the validation of the HIRS daily dataset is explained. The involved metrics are defined and the triple collocation method is explained.
3.1 Bias, root mean square difference (RMSD) and standard deviation
Standard statistical metrics are used for the validation of the dataset. When comparing two OLR maps, OLR1(x,y) and OLR2(x,y), we can define the following quantities
Mathdisplay |
---|
bias = \frac{\sum\limits_{x=1}^{180} \sum\limits_{y=1}^{360} w(x) (OLR_1(x,y) - OLR_2(x,y))}{\sum\limits_{x=1}^{180} \sum\limits_{y=1}^{360} w(x)} (1) |
Mathdisplay |
---|
RMSD = \sqrt{\frac{\sum\limits_{x=1}^{180} \sum\limits_{y=1}^{360} w(x) (OLR_1(x,y) - OLR_2(x,y))^2}{\sum\limits_{x=1}^{180} \sum\limits_{y=1}^{360} w(x)}} (2)
|
Mathdisplay |
---|
stddev = \sqrt{RMSD^2 - bias^2} (3)
|
in which the sums are done on the 180x360 grid boxes (x,y), and w(x) is a weighting proportional to the Earth surface of the 1°x1° box. The bias represents the difference in global mean OLR between the 2 datasets. The RMSD is the Root Mean Square of the difference and the standard deviation (stddev) is the RMSD corrected for the overall bias. These 3 quantities are expressed in W/m² units.
Anchor | ||||
---|---|---|---|---|
|
In [D3], intercomparisons are performed between HIRS OLR, CERES and the CM SAF GERB/SEVIRI data records. When three data sources are available, it is possible to convert the observed standard deviations between the records into individual accuracies. This is only possible under the assumption that the errors are not correlated. In [D3] the assumption about un-correlation of the errors is supported by the fact that the records are derived from totally different space instruments (respectively GERB/SEVIRI, CERES, HIRS) operated from different satellites and orbits (geostationary for MSG, polar for Aqua+Terra, polar for NOAA+MetOp).
By denoting A,B,C as the 3 data records and if the errors have normal distributions and are uncorrelated, we can write
Mathdisplay |
---|
\circ^2(A-B) = \circ^2(A) + \circ^2(B)\;(4)
|
Mathdisplay |
---|
\circ^2(A-C) = \circ^2(A) + \circ^2(C)\;(5) |
Mathdisplay |
---|
\circ^2(B-C) = \circ^2(B) + \circ^2(C)\;(6) |
where
Mathdisplay |
---|
\circ |
is the root mean square of either the difference (left terms) or of the error of the dataset. The previous relations can be inverted into:
Mathdisplay |
---|
\circ^2(A) = 0.5* (\circ^2(A-B) + \circ^2(A-C) - \circ^2(B-C))\;(7) |
Mathdisplay |
---|
\circ^2(B) = 0.5* (\circ^2(A-B) + \circ^2(B-C) - \circ^2(A-C))\;(8) |
Mathdisplay |
---|
\circ^2(C) = 0.5* (\circ^2(A-C) + \circ^2(B-C) - \circ^2(A-B))\;(9) |
The main results are presented in this document in Section 4.2.
Anchor | ||||
---|---|---|---|---|
|
Anchor | ||||
---|---|---|---|---|
|
This section summarizes the main results of the intercomparison of HIRS OLR CDR v01r02 with CERES EBAF Edition 4 as reported in [D2]. The comparison is done over the period from March 2000 to February 2018. The daily HIRS OLR products have been monthly averaged before comparison with EBAF. When comparing with CERES, it is assumed that the errors are not correlated which seems like a reasonable assumption as CERES and HIRS are totally different instruments and are flying on different orbits. The intercomparison with CERES is likely to slightly overestimate the error affecting the HIRS OLR CDR because it incorporates some uncertainty due to the CERES estimation (e.g. spatial and temporal sampling, angular dependency model).
Figure 4-1 shows the spatial variations of the mean difference between the products. In general, the HIRS OLR is slightly lower than CERES EBAF. Figure 4-2 shows the time series of the global mean difference. Although there is a clear seasonal cycle, a clear average bias exists of about 2.3 W/m² and standard deviation average value of 1.5 W/m² [D2]. This bias seems to remain stable during the 2002-2016 time period, complying with the 0.3W/m²/decade stability requirement defined by GCOS (according to [D2]). Figure 4-3 shows the spatial variations of the standard deviation (RMS of the difference corrected by the bias) between the products. The standard deviation remains stable during the 2002-2016 time period, but shows slightly higher values before July 2002. This is likely due to degraded CERES products before July 2002, when they can use only CERES data from the Terra satellite.
Figure 4-1: Mean difference between daily mean HIRS OLR v01r02 and CERES EBAF Ed 4.0, over the period March 2000 - February 2018 (extracted from [D2], Fig. 21 left). Anchor figure4_1 figure4_1
Figure 4-2: Timeseries of global mean monthly OLR difference (red), the standard deviation of the differences (cyan), and the RMS difference (blue) and between monthly-integrated v01r02 Daily OLR CDR and CERES EBAF Ed 4.0 (extracted from [D2], Fig. 20). Anchor figure4_2 figure4_2
Figure 4-3: Standard deviation of the difference between daily mean HIRS OLR v01r02 and CERES EBAF Ed 4.0 over the period March 2000 – December 2016 (extracted from [D2], Fig. 21 right). Anchor figure4_3 figure4_3
Anchor | ||||
---|---|---|---|---|
|
We have applied the triple collocation method (see Section 3.2) to the all sky monthly mean OLR data from CM SAF, CERES SYN and HIRS. The assumption about no correlation between the errors is supported by the fact that the records are derived from different instruments (respectively GERB/SEVIRI, CERES, HIRS) operated from different satellites and orbits (geostationary for MSG, polar for Aqua+Terra, polar for NOAA+MetOp).
The result is shown in Figure 4. The averaged accuracies are estimated at: 1.6 W/m² for CM SAF and 0.9 W/m² for SYN MM (Monthly Mean) and 0.9 W/m² for HIRS v01r02 monthly mean. Note that these numbers are estimated over the 60°N-60°S and 60°W-60°E region and can therefore differ from the global mean numbers given in Section 4.1.
Figure 4-4: Individual accuracies of the monthly mean OLR (here called TOA Emitted Thermal TET) from CM SAF GERB/SEVIRI ed02, CERES SYN1deg-month ed4.0 and HIRS v01r02, as estimated from their RMS differences (extracted from [D4]). Anchor figure4_4 figure4_4
Anchor | ||||
---|---|---|---|---|
|
Figure 4-5 shows the global mean bias between HIRS v01r02 MM and several of the other CDRs described in Section 2. In general, the HIRS OLR record looks stable but the following is worth noting for possible further investigation/reprocessing:
From 2018 onward, the HIRS OLR seems to decrease with respect to the CERES, CLARA and ERA5. This is further discussed in the next section.
A couple of months around December 2000 seem to be affected by higher negative bias.
Except for this, there is no evidence of jumps or drifts that would affect the HIRS OLR v01r02 CDR.
Figure 4-5: Bias of the HIRS OLR v01r02 with respect to CERES EBAF, CERES SYN, ERA5, ISCCP, CLOUD CCI and CLARA-A3. Anchor figure4_5 figure4_5
4.4 Recent data evaluation (as of February 2022)
This section addresses the evaluation of the recent data, including the interim (ICDR) data. Figure 4-6 shows the global monthly mean difference between HIRS OLR v01r02 and CERES EBAF ed4.1 (that was available until November 2021 at time of writing). The 2.5% and 97.5% percentiles of the bias have been determined on the “final” part of the CDR. Figure 4-7: Standard deviation between HIRS OLR v01r02 and CERES EBAF d4.1. From 2018 onward, a decrease of OLR is well visible and is attributed to the orbital drift of the afternoon satellites (NOAA18 and NOAA19). The maximum negative bias seemed to have occurred in January 2021 and the level has been recovering during 2021. This stability problem is expected to be removed in a further release of the HIRS OLR CDR that will ingest also CrIS and IASI data for the satellites without HIRS instrument (NPP, NOAA20 and successors, Metop-C).
Figure 4-7 shows the standard deviation between the products. The higher uncertainty between March 2000 and July 2003 is explained by the absence of CERES afternoon observations. From 2018 onward, the slight increase of the standard deviation is attributed to the NOAA18 and NOAA19 orbit drift.
Figure 4-6: Global monthly mean OLR difference between HIRS OLR v01r02 and CERES EBAF ed4.1. The 2.5% and 97.5% percentiles are evaluated on the “final” data (up to 31/12/2020). Anchor figure4_6 figure4_6
Figure 4-7: Standard deviation between HIRS OLR v01r02 and CERES EBAF d4.1. Anchor figure4_7 figure4_7
4.5 Summary
4.5.1 Accuracy
Using CERES EBAF as the reference, the accuracy (standard deviation) of the HIRS OLR product is evaluated at 1.5 W/m² [D2]. A part of this value could however be attributed to errors in the CERES products, so the uncertainty in the monthly mean HIRS OLR values is likely better than 1.5 W/m². Assuming a similar uncertainty level for the HIRS OLR and CERES products, the observed 1.5 W/m² standard deviation would reduce to 0.9 W/m² uncertainties for both products. Such a level of uncertainty is supported by the triple collocation of CERES, HIRS and GERB OLR.
4.5.2 Stability
The HIRS OLR record shows good stability with respect to CERES but a drift towards more negative bias is observed after 2018. Stability over the pre-CERES era is more difficult to assess due to lack of a stable OLR reference. However, the timeseries of anomalies with respect to ERA5, CLARA-A3, Cloud CCI, ISCCP-FH products do not show any obvious stability problem.
Anchor | ||||
---|---|---|---|---|
|
Clerbaux, N., Urbain, M., Ipe, A., Baudrez, E., Velazquez-Blazquez, A., Akkermans, T., Hollmann, R., Fuchs, P., Selbach, N., Werscheck, M., 2017: CM SAF TOA Radiation GERB/SEVIRI Data Record - Edition 2, Satellite Application Facility on Climate Monitoring, DOI:10.5676/EUM_SAF_CM/TOA_GERB/V002.
Doelling, D. R., M. Sun, L. T. Nguyen, M. L. Nordeen, C. O. Haney, D. F. Keyes, P. E. Mlynczak, 2016: Advances in Geostationary-Derived Longwave Fluxes for the CERES Synoptic (SYN1deg) Product, Journal of Atmospheric and Oceanic Technology, 33(3), 503-521. doi: 10.1175/JTECH-D-15-0147.1
Doelling, D. R., N. G. Loeb, D. F. Keyes, M. L. Nordeen, D. Morstad, C. Nguyen, B. A. Wielicki, D. F. Young, M. Sun, 2013: Geostationary Enhanced Temporal Interpolation for CERES Flux Products, Journal of Atmospheric and Oceanic Technology, 30(6), 1072-1090. doi: 10.1175/JTECH-D-12-00136.1.
Hersbach, H., Bell, B., Berrisford, P., Hirahara, S., Horányi, A., Muñoz‐Sabater, J., Nicolas, J., Peubey, C., Radu, R., Schepers, D. and Simmons, A., 2020: The ERA5 global reanalysis. Quarterly Journal of the Royal Meteorological Society, 146(730), pp.1999-2049.
Hogan, R.J. and Bozzo, A., 2018. A flexible and efficient radiation scheme for the ECMWF model. Journal of Advances in Modeling Earth Systems, 10(8), pp.1990-2008.
Knapp, K. R., S. Ansari, C. L. Bain, M. A. Bourassa, M. J. Dickinson, C. Funk, C. N. Helms, C. C. Hennon, C. D. Holmes, G. J. Huffman, J. P. Kossin, H.-T. Lee, A. Loew, and G. Magnusdottir, 2011: Globally gridded satellite (GridSat) observations for climate studies. Bulletin of the American Meteorological Society, 92, 893-907. doi:10.1175/2011BAMS3039.1
Lee, H.-T.; Heidinger, A.; Gruber, A.; Ellingson, R.G.: The HIRS outgoing longwave radiation product from hybrid polar and geosynchronous satellite observations. Adv. Space Res. 2004, 33, 1120–1124
Lee, H.-T., 2014: Daily OLR CDR – Development and Evaluation. CERES Science Team Meeting, Apr 2014
Lee, H.-T., 2014: Daily OLR Climate Data Record. EGU General Assembly, Apr 2014
Lee, H.-T., 2018: Quality Assurance Summary and Results for Monthly and Daily OLR CDR (rev.20180831).
Lee, H.-T., A. Gruber, R. G. Ellingson and I. Laszlo, 2007: Development of the HIRS Outgoing Longwave Radiation climate data set. J. Atmos. Ocean. Tech., 24, 2029–2047.
Lee, H.-T., C. J. Schreck, and K. R. Knapp, 2014: Generation of Daily OLR CDR. Eumetsat Meteorological Satellite Conference, Sep 2014
Loeb, N. G., D. R. Doelling, H. Wang, W. Su, C. Nguyen, J. G. Corbett, L. Liang, C. Mitrescu, F. G. Rose, and S. Kato, 2018: Clouds and the Earth’s Radiant Energy System (CERES) Energy Balanced and Filled (EBAF) Top-of-Atmosphere (TOA) Edition-4.0 Data Product. J. Climate, 31, 895-918, doi: 10.1175/JCLI-D-17-0208.1
Loeb, N. G., S. Kato, and B. A. Wielicki, 2002: Defining top-of-atmosphere flux reference level for Earth Radiation Budget studies. J. Climate, 15, 3301-3309.
Schreck, C. J., H.-T. Lee and K. Knapp, 2018: HIRS Outgoing Longwave Radiation—Daily Climate Data Record: Application toward Identifying Tropical Subseasonal Variability. Remote Sens. 2018, 10, 1325
Schulz, J., Albert, P., Behr, H.D., Caprion, D., Deneke, H., Dewitte, S., Durr, B., Fuchs, P., Gratzki, A., Hechler, P. and Hollmann, R., 2009: Operational climate monitoring from space: the EUMETSAT Satellite Application Facility on Climate Monitoring (CM-SAF). Atmospheric Chemistry and Physics, 9(5), pp.1687-1709.
Stengel, M., Stapelberg, S., Sus, O., Finkensieper, S., Würzler, B., Philipp, D., Hollmann, R., Poulsen, C., Christensen, M. and McGarragh, G., 2020: Cloud_cci Advanced Very High Resolution Radiometer post meridiem (AVHRR-PM) dataset version 3: 35-year climatology of global cloud and radiation properties. Earth System Science Data, 12(1), pp.41-60.
Wielicki, B.A., Barkstrom, B.R., Harrison, E.F., Lee III, R.B., Smith, G.L. and Cooper, J.E., 1996. Clouds and the Earth's Radiant Energy System (CERES): An earth observing system experiment. Bulletin of the American Meteorological Society, 77(5), pp.853-868.
Young, A. H., Knapp, K. R., Inamdar, A., Hankins, W., and Rossow, W. B., 2018: The International Satellite Cloud Climatology Project H-Series climate data record product, Earth Syst. Sci. Data, 10, 583–593, doi:10.5194/essd-10-583-2018.
...
...
We have applied the triple collocation method to the all sky monthly mean OLR data from CM SAF, CERES SYN and HIRS. The assumption about no existing correlation between the errors is supported by the fact that the data records are derived from different space instruments (respectively GERB/SEVIRI, CERES, HIRS) operated from different satellites/orbits (geostationary for MSG, polar for Aqua+Terra, polar for NOAA+MetOp).
The result is shown in Figure 4-5. The estimated accuracies are on average: 1.6 W/m² for CM SAF and 0.9 W/m² for SYN MM (Monthly Mean) and HIRS MM. Note that those numbers are estimated over the 60°N-60°S and 60°W-60°E region and cannot be directly compared with the comparison given in section 4.1.
...
4.3 Comparison with reanalysis
The representation of radiative fluxes has significantly improved in recent reanalysis like ERA5. Although there still exist important differences with observed TOA radiation, reanalysis is however helpful to evaluate stability and detect possible jumps and artifacts in the time series.
4.4 Application(s) specific assessments
In addition to the extensive product validation (see chapter 3 for results and chapter 2/3 in [D4] for validation methodology) a second assessment is introduced to evaluate the Interim Climate Data Record (ICDR) against the Thematic Climate Data Record (TCDR) in terms of consistency. Since frequent ICDR deliveries make detailed validation not feasible, a consistency check against the deeply validated TCDR is used as an indication of quality. This is done by a comparison of the following two evaluations:
- TCDR against a stable, long-term and independent reference dataset
- ICDR against the same stable, long-term and independent reference dataset
The evaluation method is generated to detect differences in the ICDR performance in a quantitative, binary way with so called Key Performance Indicators. The general method is outlined in [D4] chapter 3. The same difference between TCDR/ICDR and the reference dataset would lead to the conclusion that TCDR and ICDR have the same quality (key performance is "good"). Variations or trends in the differences (TCDR/ICDR against reference) would require a further investigation to analyze the reasons. The key performance would be marked as "bad". The binary decision whether the key performance is good or bad is made in a statistical way by a hypotheses test (binomial test). Based on the TCDR/reference comparison (global means, monthly or daily means) a range is defined with 95% of the differences are within. This range (2.5 and 97.5 percentile) is used for the ICDR/reference comparison to check whether the values are in or out of the range. The results could be the following:
- All or a sufficient high number of ICDR/reference differences lies within the range defined by the TCDR/reference comparison: Key performance of the ICDR is "good"
- A smaller number of ICDR/reference differences is within the pre-defined range: Key performance of the ICDR is "bad"
4.4.1 Results
The results of the KPI test are summarized in Table 4-1.
...
p2.5 = -5.35 W/m²
p97.5 = -3.02 W/m²
...
9/9
...
5/5
...
Reference data not available
...
8/8
Percentiles were calculated based on the comparison of the TCDR using the High Resolution Infrared Radiation Sounder (HIRS) instrument onboard NOAA and Metop satellites against ERA5 data for the variable Outgoing Longwave Radiation (OLR). Reference period is defined from 01/1979 to 12/2020. The complete ICDR remains within the limits leading to "good" KPI tests. The ICDR is therefore stable in relation to the deeply validated TCDR.
4.5 Discussion
Using CERES EBAF as the reference, the accuracy of the HIRS OLR product is evaluated at 1.8 W/m² [D5]. A part of this value could however be attributed to errors in the CERES products, so the uncertainty in the monthly mean HIRS OLR values is possibly better than 1.8 W/m². Assuming a similar uncertainty for HIRS and CERES products, the observed 1.8 W/m² standard deviation would reduce to 1.3 W/m² uncertainties for both products.
References
Clerbaux, Nicolas; Urbain, Manon; Ipe, Alessandro; Baudrez, Edward; Velazquez-Blazquez, Almudena; Akkermans, Tom; Hollmann, Rainer; Fuchs, Petra; Selbach, Nathalie; Werscheck, Martin, 2017: CM SAF TOA Radiation GERB/SEVIRI Data Record - Edition 2, Satellite Application Facility on Climate Monitoring, DOI:10.5676/EUM_SAF_CM/TOA_GERB/V002.
Doelling, D. R., M. Sun, L. T. Nguyen, M. L. Nordeen, C. O. Haney, D. F. Keyes, P. E. Mlynczak, 2016: Advances in Geostationary-Derived Longwave Fluxes for the CERES Synoptic (SYN1deg) Product, Journal of Atmospheric and Oceanic Technology, 33(3), 503-521. doi: 10.1175/JTECH-D-15-0147.1
Doelling, D. R., N. G. Loeb, D. F. Keyes, M. L. Nordeen, D. Morstad, C. Nguyen, B. A. Wielicki, D. F. Young, M. Sun, 2013: Geostationary Enhanced Temporal Interpolation for CERES Flux Products, Journal of Atmospheric and Oceanic Technology, 30(6), 1072-1090. doi: 10.1175/JTECH-D-12-00136.1.
Knapp, K. R., S. Ansari, C. L. Bain, M. A. Bourassa, M. J. Dickinson, C. Funk, C. N. Helms, C. C. Hennon, C. D. Holmes, G. J. Huffman, J. P. Kossin, H.-T. Lee, A. Loew, and G. Magnusdottir, 2011: Globally gridded satellite (GridSat) observations for climate studies. Bulletin of the American Meteorological Society, 92, 893-907. doi:10.1175/2011BAMS3039.1
Lee, H.-T.; Heidinger, A.; Gruber, A.; Ellingson, R.G. The HIRS outgoing longwave radiation product from hybrid polar and geosynchronous satellite observations. Adv. Space Res. 2004, 33, 1120–1124
Lee, H.-T., 2014: Daily OLR CDR – Development and Evaluation. CERES Science Team Meeting, Apr 2014
Lee, H.-T., 2014: Daily OLR Climate Data Record. EGU General Assembly, Apr 2014
Lee, H.-T., 2018: Quality Assurance Summary and Results for Monthly and Daily OLR CDR (rev.20180831).
Lee, H.-T., A. Gruber, R. G. Ellingson and I. Laszlo, 2007: Development of the HIRS Outgoing Longwave Radiation climate data set. J. Atmos. Ocean. Tech., 24, 2029–2047.
Lee, H.-T., C. J. Schreck, and K. R. Knapp, 2014: Generation of Daily OLR CDR. Eumetsat Meteorological Satellite Conference, Sep 2014
Loeb, N. G., D. R. Doelling, H. Wang, W. Su, C. Nguyen, J. G. Corbett, L. Liang, C. Mitrescu, F. G. Rose, and S. Kato, 2018: Clouds and the Earth's Radiant Energy System (CERES) Energy Balanced and Filled (EBAF) Top-of-Atmosphere (TOA) Edition-4.0 Data Product. J. Climate, 31, 895-918, doi: 10.1175/JCLI-D-17-0208.1
Loeb, N. G., S. Kato, and B. A. Wielicki, 2002: Defining top-of-atmosphere flux reference level for Earth Radiation Budget studies. J. Climate, 15, 3301-3309.
...
Info | ||
---|---|---|
| ||
Related articles
Content by Label | ||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|