Contributors: Rory Gray (Met Office)

Issued by: Met Office / Rory Gray
Date: 30/11/2020
Ref: C3S_D312b_Lot1.2.3.11-v1.2_202011_PQAR_ECV_WVP_UTH-ICDR_v1.0.docx
Official reference number service contract: C3S_312b_Lot1_DWD/SC1

Table of Contents

History of modifications

Version

Date

Description of modification

Chapters / Sections

1.0

30/11/2020

Initial version (based on D2.3.11-v1.1 updated to include performance up to
31/8/2020)

All

List of datasets covered by this document

Deliverable ID

Product title

Product type (CDR, ICDR)

Version number

Delivery date

D3.3.12-v1.1P3

Water Vapour UTH_MW ICDR v1.1 (Quarterly) – Period 3

ICDR

1.1

08/2019

D3.3.12-v1.2P4

Water Vapour UTH_MW ICDR v1.2 (Quarterly) – Period 4

ICDR

1.2

11/2019

D3.3.12-v1.3P4

Water Vapour UTH_MW ICDR v1.3 (Quarterly) – Period 4

ICDR

1.3

02/2020

D3.3.12-v1.4P5

Water Vapour UTH_MW ICDR v1.4 (Quarterly) – Period 5

ICDR

1.4

05/2020

D3.3.12-v1.5P5

Water Vapour UTH_MW ICDR v1.5 (Quarterly) – Period 5

ICDR

1.5

08/2020

Related documents

Reference ID

Document

D1

Gray, R., Tsamalis, C., Saunders, R. (2020) C3S Water Vapour

Service: Product Quality Assurance Document. Copernicus Climate Change Service,

Document ref. C3S_D312b_Lot1.2.3.10-v1.2_202008_PQAD_ECV_WVP_UTH_ICDR_v1.0.1

UTH ICDR: Product Quality Assurance Document (PQAD)

Last accessed on 01.09.2023

D2

Meirink, J.F., et al, (2023) C3S cross ECV document

Service: Key Performance Indicators (KPIs). Copernicus Climate Change Service,

Document ref. C3S2_D312a_Lot1.3.7.1_202303_Unified_KPI_Approach_v1.0

Key Performance Indicators (KPIs)

Last accessed: 23.08.2023

Acronyms

Acronym

Definition

AMSU-B

Advanced Microwave Sounding Unit-B

C3S

Copernicus Climate Change Service

CM SAF

Climate Monitoring Satellite Application Facility

ECMWF

European Centre for Medium-Range Weather Forecasts

EUMETSAT

European Organisation for the Exploitation of Meteorological Satellites

ICDR

Interim Climate Data Record

KPI

Key Performance Indicator

MHS

Microwave Humidity Sounder

MW

Microwave

NOAA

National Oceanic and Atmospheric Administration

PQAD

Product Quality Assurance Document

PQAR

Product Quality Assessment Report

TCDR

Thematic Climate Data Record

UTH

Upper Tropospheric Humidity

List of tables

Table 3-1: Summary of KPI results with 2.5 and 97.5 percentiles and number of ICDR months within the range

Table 4-1: showing the number of daily mean ICDR UTH – reanalysis derived UTH daily mean differences that fall within the defined KPI values (-0.47% to 0.55% (1/1/2016 to 31/12/2018), and 0.40% to 1.57% (1/1/2019 to 31/8/2020) 

Table 4-2: showing the number of daily mean ICDR UTH – reanalysis derived UTH daily mean differences that fall within the TCDR 2.5 and 97.5 percentiles for each satellite

List of figures

Figure 4-1: Global daily mean ICDR UTH – ERA-Interim derived UTH differences, within +/-60 degrees latitude, for NOAA-18, 1/1/2016-20/10/2018

Figure 4-2: Number of daily data matchups used to determine the data shown in Figure 4-1 month (2016-2020)

Figure 4-3: Global daily mean ICDR UTH – ERA Interim (to 31/12/2018), and ICDR UTH – ERA5 (from 1/1/2019) UTH differences, within +/-60 degrees latitude, for MetOp-A, 1/1/2016-31/8/2020 month (2016-2020)

Figure 4-4: Number of daily data matchups used to determine the data shown in Figure 4-3

Figure 4-5: Global daily mean ICDR UTH – ERA Interim (to 31/12/2018), and ICDR UTH – ERA5 (from 1/1/2019)ERA-Interim derived UTH differences, within +/-60 degrees latitude, for MetOp-B,1/1/2016-31/8/2020 month (2016-2020)

Figure 4-6: Number of daily data matchups used to determine the data shown in Figure 4-5

Scope of the document

This document is the Product Quality Assurance Report (PQAR), version 1.2, for the microwave upper tropospheric humidity (UTH) ICDR product. It summarises the results from the product assessment described in the Product Quality Assurance Document (PQAD) [D1].

The product validation methodology is described in the aforementioned PQAD with results shown in detail in this document.

Executive summary

This report concerns the results of the validation of the C3S Upper Tropospheric Humidity (UTH) interim climate data record (ICDR) derived from the Advanced Microwave Sounding Unit-B (AMSU-B) on-board the National Oceanic and Atmospheric Administration (NOAA) satellite NOAA-18 (up to 20/10/2018) (https://www.wmo-sat.info/oscar/satellites/view/340) and Microwave Humidity Sounder (MHS) on-board the MetOp satellites (A and B) (https://www.wmo-sat.info/oscar/satellites/view/306, https://www.wmo-sat.info/oscar/satellites/view/307), from January 2016 to August 2020. 

Until the end of 2018, the ICDR has been validated against the equivalent UTH derived from ECMWF’s ERA-Interim global atmospheric reanalysis (https://www.ecmwf.int/en/forecasts/datasets/reanalysis-datasets/era-interim) in the same way as was done for the TCDR equivalent product, details of which are referenced in the PQAD [D1]. From 1/1/2019 onwards, the ICDR has been validated against the equivalent UTH derived from ECMWF’s ERA-5 global atmospheric reanalysis (https://www.ecmwf.int/en/forecasts/datasets/reanalysis-datasets/era5), as explained in [D1].

Key Performance Indicators (KPIs) were defined in terms of deviations of the UTH CDR from the ERAInterim (until 31/12/2018 inclusive) and ERA-5 (from 1/1/2019 onwards) reanalysis reference data sets, by the methods described in [D2, section 3], as specified in [D1]. 

Comparison of the ICDR dataset against the reference dataset using the defined KPIs confirms that, for MetOp-A,-B and NOAA-18, until 31/12/2018 the ICDR dataset differs from the reference dataset in the same way that the TCDR dataset does, and since 1/1/2019 the ICDR dataset differs from the ERA-5 reference dataset in the same way that it did throughout 2018.

1. Product validation methodology

The product assessed in this report is the microwave upper tropospheric humidity (UTH), which is based on observations from the microwave (MW) sounders Advanced Microwave Sounding Unit–B (AMSU-B) and Microwave Humidity Sounder (MHS), on board the satellite platforms NOAA-18 and MetOp (A and B). For a full description of the validation methodology refer to the UTH ICDR PQAD [D1]. The relevant MHS channel on NOAA-19 has had noise issues since 2009 [Hans et al., 2017], and consequently was included in neither the derivation of the KPI values nor this product assessment.

2. Validation results

The results of the comparison between the ICDR and the ERA-Interim/ERA-5 UTH are presented in this section. The validation results show that the ICDR varies against the ERA-Interim derived equivalent UTH in the same way as the TCDR does, and varies against the ERA-5 derived equivalent UTH in the same way since 1/1/2019 as it did throughout 2018, and remains in agreement with the established KPIs, a new set of which were established with reference to ERA-5 to be used for data since 1/1/2019, as explained in [D1] and [D2].

If the absolute difference between the ICDR and the reanalysis UTH is greater than 50% these points are excluded from the subsequent analysis. The majority of these excluded points are found close to the poles, where both the channel 183.31±1 GHz does not truly sense the upper troposphere, and the emissivity over the ice used in the simulations has significant uncertainties. A threshold of 50% is deemed reasonable as any valid differences should be significantly smaller than this value, and its use should therefore eliminate really problematic situations for either datasets without significantly affecting the statistical results. Finally, although ERA-Interim and ERA-5 are used here as the reference data sets, like all data sets, they also have uncertainties.

Figures 4-1 - 4/6 present the time series of the daily mean difference (ICDR minus reanalysis) and the number of matched 1 degree gridboxes between the two datasets for the three satellites and the zonal region within ±60 degrees latitude, over the time range of the ICDR dataset, for each of NOAA18, MetOp-A and MetOp-B. In each case the performance with respect to the 97.5 and 2.5 percentiles of UTH – ERA-Interim (until 31/12/2018) and UTH – ERA-5 (from 1/1/2019) derived UTH differences is consistent with the KPIs, specified in [D1] and [D2, Table 3] as 0.55% (97.5 percentile) and -0.47% (2.5 percentile) until 31/12/2018, and 1.57% (97.5 percentile) and 0.40% (2.5 percentile) since 1/1/2019. Outlying UTH difference values occur on days for which the number of data points is very low.

Table 1 shows the number of global (within +/-60degree latitude) daily mean UTH differences falling within and outside the KPI values of 0.55% (1.40%) and -0.47% (0.40%). Figures 4-1 - 4/6 show that many of the outliers occur on days when there are significantly fewer than usual valid differences available. For both MetOp satellites, the differences are within the KPI values approximately 95% of the time until 31/12/2018, similarly as for the TCDRs from which the KPIs were derived, and more than 90% of the time since 1/1/2019. The NOAA-18 values are within the KPIs more than 93% of the time, with the upper, 0.55%, KPI being violated 4.7% of the time, in particular over a period in early 2018 when the number of valid data points available remained lower than normal (see Figure 4-2). NOAA-18 ceased providing data in October 2018.

Table 2 compares the performance in terms of the 2.5 and 97.5 percentiles for each satellite separately. Approximately 95% of the ICDR UTH differences for both MetOp satellites are within the 2.5 and 97.5 percentiles, except for MetOp-A at approximately 90% since 1/1/2019 due to 8.4% violations of the upper KPI since 1/1/2019. Around 90% of the NOAA-18 ICDR differences are within the 2.5 and 97.5 percentiles its TCDR, with most of the violations occurring during times of reduced data.

3. Application(s) specific assessments

All specific assessments performed are made in terms of compliance with the KPIs derived as described in reference document [D2].

4. Compliance with user requirements

The fulfillment of the derived KPIs (see [D2]) in terms of performance against the reference dataset has been demonstrated during periods of good data coverage.

Figures and Tables

Figure 4-1: Global daily mean ICDR UTH – ERA-Interim derived UTH differences, within +/-60 degrees latitude, for NOAA-18, 1/1/2016-20/10/2018.

Figure 4-2: Number of daily data matchups used to determine the data shown in Figure 4-1 month (2016-2020).

Figure 4-3: Global daily mean ICDR UTH – ERA Interim (to 31/12/2018), and ICDR UTH – ERA5 (from 1/1/2019) UTH differences, within +/-60 degrees latitude, for MetOp-A, 1/1/2016-31/8/2020 month (2016-2020).

Figure 4-4: Number of daily data matchups used to determine the data shown in Figure 4-3.

Figure 4-5: Global daily mean ICDR UTH – ERA Interim (to 31/12/2018), and ICDR UTH – ERA5 (from 1/1/2019)ERA-Interim derived UTH differences, within +/-60 degrees latitude, for MetOp-B,1/1/2016-31/8/2020 month (2016-2020)

Figure 4-6: Number of daily data matchups used to determine the data shown in Figure 4-5


Table 4-1: Showing the number of daily mean ICDR UTH – reanalysis derived UTH daily mean differences that fall within the defined KPI values (-0.47% to 0.55% (1/1/2016 to 31/12/2018), and 0.40% to 1.57% (1/1/2019 to 31/8/2020). For MetOp-A and MetOp-B the performance against each of the ERA-interim (2016-2018) and ERA-5 (2019-2020) derived KPI limits are shown separately.


Number of daily global (+/-60deg) mean UTH differences

Number of daily mean global (+/-60deg) UTH differences

> \( \Big\{\begin{array}{ll} 0.55\% (1/1/2016 - 31/12/2018) \\ 1.57\% (1/1/2019 − 31/8/2020)\end{array} \)



Number of daily mean global (+/-60deg) UTH differences

< \( \Big\{\begin{array}{ll} -0.47\% (1/1/2016 - 31/12/2018) \\ 0.40\% (1/1/2019 − 31/8/2020)\end{array} \)


Percentage of
UTH daily mean global (+/-60deg) UTH differences
within the KPI values

KPI reanalysis baseline

NOAA-18
(1/1/2016 - 20/10/2018)

1023

48 (4.7%)

21 (2.1%)

954 (93.2%)

ERAinterim

Metop-A \( \Big\{\begin{array}{ll} 2016 - 2018 \\ \textbf{2019 - 2020}\end{array} \)

1095
607

32 (2.9%)
51 (8.4%)

3 (0.3%)
5 (0.8%)

1060 (96,8%)
551 (90,8%)

ERAinterim
ERA5

Metop- B \( \Big\{\begin{array}{ll} 2016 - 2018 \\ \textbf{2019 - 2020}\end{array} \)

1092
606

5 (0.5%)
2 (0.3%)

42 (3.8%)
3 (0.5%)

1045 (95.7%)
601 (99.2%)

ERAinterim
ERA5


Table 4-2: showing the number of daily mean ICDR UTH – reanalysis derived UTH daily mean differences that fall within the TCDR 2.5 and 97.5 percentiles for each satellite. For MetOp-A and MetOp-B the performance against each of the ERA-interim (2016-2018) and ERA-5 (2019-2020) derived KPI limits are shown separately.


2.5 percentile KPI

97.5 percentile KPI

Number of daily mean ICDR UTH diffs

Number of ICDR daily mean differences < TCDR 2.5 percentile

Number of ICDR daily mean differences > TCDR 97.5 percentile

Number of ICDR daily mean differences within TCDR 2.5 and 97.5 percentiles

KPI reanalysis baseline



NOAA-18

-0.46

0.47

1023

22 (2.2%)

83 (8.1%)

918 (89.7%)

ERAinterim

Metop-A \( \Big\{\begin{array}{ll} 2016 - 2018 \\ \textbf{2019 - 2020}\end{array} \)

-0.39
0.68

0.55
1.57

1095
607

8 (0.7%)
9 (1.5%)

32 (2.9%)
51 (8.4%)

1055 (96.3%)
547 (90.1%)

ERAinterim
ERA5

Metop-B \( \Big\{\begin{array}{ll} 2016 - 2018 \\ \textbf{2019 - 2020}\end{array} \)

-0.47
0.40

0.46
1.35

1092
606

42 (3.8%)
3 (0.5%)

16 (1.5%)
32 (5.3%)

1034 (94.7%)
571 (94.2%)

ERAinterim
ERA5

References

Hans, I.; Burgdorf, M.; John, V.O.; Mittaz, J.; Buehler, S.A. Noise performance of microwave humidity sounders over their lifetime. Atmos. Meas. Tech. 2017, 10, 4927–4945.

This document has been produced with funding by the European Union in the context of the Copernicus Climate Change Service (C3S), operated by the European Centre for Medium-Range Weather Forecasts on behalf on the European Union (Contribution Agreement signed on 22/07/2021).

All information in this document is provided "as is" and no guarantee of warranty is given that the information is fit for any particular purpose.

The users thereof use the information at their sole risk and liability. For the avoidance of all doubt, the European Commission and the European Centre for Medium-Range Weather Forecasts have no liability in respect of this document, which is merely representing the author's view.

Related articles