Contributors: Rory Gray (Met Office)
Issued by: Met Office / Rory Gray
Date: 30/11/2020
Ref: C3S_D312b_Lot1.2.3.11-v1.2_202011_PQAR_ECV_WVP_UTH-ICDR_v1.0.docx
Official reference number service contract: C3S_312b_Lot1_DWD/SC1
History of modifications
List of datasets covered by this document
Related documents
Acronyms
List of tables
List of figures
Scope of the document
This document is the Product Quality Assurance Report (PQAR), version 1.2, for the microwave upper tropospheric humidity (UTH) ICDR product. It summarises the results from the product assessment described in the Product Quality Assurance Document (PQAD) [D1].
The product validation methodology is described in the aforementioned PQAD with results shown in detail in this document.
Executive summary
This report concerns the results of the validation of the C3S Upper Tropospheric Humidity (UTH) interim climate data record (ICDR) derived from the Advanced Microwave Sounding Unit-B (AMSU-B) on-board the National Oceanic and Atmospheric Administration (NOAA) satellite NOAA-18 (up to 20/10/2018) (https://www.wmo-sat.info/oscar/satellites/view/340) and Microwave Humidity Sounder (MHS) on-board the MetOp satellites (A and B) (https://www.wmo-sat.info/oscar/satellites/view/306, https://www.wmo-sat.info/oscar/satellites/view/307), from January 2016 to August 2020.
Until the end of 2018, the ICDR has been validated against the equivalent UTH derived from ECMWF’s ERA-Interim global atmospheric reanalysis (https://www.ecmwf.int/en/forecasts/datasets/reanalysis-datasets/era-interim) in the same way as was done for the TCDR equivalent product, details of which are referenced in the PQAD [D1]. From 1/1/2019 onwards, the ICDR has been validated against the equivalent UTH derived from ECMWF’s ERA-5 global atmospheric reanalysis (https://www.ecmwf.int/en/forecasts/datasets/reanalysis-datasets/era5), as explained in [D1].
Key Performance Indicators (KPIs) were defined in terms of deviations of the UTH CDR from the ERAInterim (until 31/12/2018 inclusive) and ERA-5 (from 1/1/2019 onwards) reanalysis reference data sets, by the methods described in [D2, section 3], as specified in [D1].
Comparison of the ICDR dataset against the reference dataset using the defined KPIs confirms that, for MetOp-A,-B and NOAA-18, until 31/12/2018 the ICDR dataset differs from the reference dataset in the same way that the TCDR dataset does, and since 1/1/2019 the ICDR dataset differs from the ERA-5 reference dataset in the same way that it did throughout 2018.
1. Product validation methodology
The product assessed in this report is the microwave upper tropospheric humidity (UTH), which is based on observations from the microwave (MW) sounders Advanced Microwave Sounding Unit–B (AMSU-B) and Microwave Humidity Sounder (MHS), on board the satellite platforms NOAA-18 and MetOp (A and B). For a full description of the validation methodology refer to the UTH ICDR PQAD [D1]. The relevant MHS channel on NOAA-19 has had noise issues since 2009 [Hans et al., 2017], and consequently was included in neither the derivation of the KPI values nor this product assessment.
2. Validation results
The results of the comparison between the ICDR and the ERA-Interim/ERA-5 UTH are presented in this section. The validation results show that the ICDR varies against the ERA-Interim derived equivalent UTH in the same way as the TCDR does, and varies against the ERA-5 derived equivalent UTH in the same way since 1/1/2019 as it did throughout 2018, and remains in agreement with the established KPIs, a new set of which were established with reference to ERA-5 to be used for data since 1/1/2019, as explained in [D1] and [D2].
If the absolute difference between the ICDR and the reanalysis UTH is greater than 50% these points are excluded from the subsequent analysis. The majority of these excluded points are found close to the poles, where both the channel 183.31±1 GHz does not truly sense the upper troposphere, and the emissivity over the ice used in the simulations has significant uncertainties. A threshold of 50% is deemed reasonable as any valid differences should be significantly smaller than this value, and its use should therefore eliminate really problematic situations for either datasets without significantly affecting the statistical results. Finally, although ERA-Interim and ERA-5 are used here as the reference data sets, like all data sets, they also have uncertainties.
Figures 4-1 - 4/6 present the time series of the daily mean difference (ICDR minus reanalysis) and the number of matched 1 degree gridboxes between the two datasets for the three satellites and the zonal region within ±60 degrees latitude, over the time range of the ICDR dataset, for each of NOAA18, MetOp-A and MetOp-B. In each case the performance with respect to the 97.5 and 2.5 percentiles of UTH – ERA-Interim (until 31/12/2018) and UTH – ERA-5 (from 1/1/2019) derived UTH differences is consistent with the KPIs, specified in [D1] and [D2, Table 3] as 0.55% (97.5 percentile) and -0.47% (2.5 percentile) until 31/12/2018, and 1.57% (97.5 percentile) and 0.40% (2.5 percentile) since 1/1/2019. Outlying UTH difference values occur on days for which the number of data points is very low.
Table 1 shows the number of global (within +/-60degree latitude) daily mean UTH differences falling within and outside the KPI values of 0.55% (1.40%) and -0.47% (0.40%). Figures 4-1 - 4/6 show that many of the outliers occur on days when there are significantly fewer than usual valid differences available. For both MetOp satellites, the differences are within the KPI values approximately 95% of the time until 31/12/2018, similarly as for the TCDRs from which the KPIs were derived, and more than 90% of the time since 1/1/2019. The NOAA-18 values are within the KPIs more than 93% of the time, with the upper, 0.55%, KPI being violated 4.7% of the time, in particular over a period in early 2018 when the number of valid data points available remained lower than normal (see Figure 4-2). NOAA-18 ceased providing data in October 2018.
Table 2 compares the performance in terms of the 2.5 and 97.5 percentiles for each satellite separately. Approximately 95% of the ICDR UTH differences for both MetOp satellites are within the 2.5 and 97.5 percentiles, except for MetOp-A at approximately 90% since 1/1/2019 due to 8.4% violations of the upper KPI since 1/1/2019. Around 90% of the NOAA-18 ICDR differences are within the 2.5 and 97.5 percentiles its TCDR, with most of the violations occurring during times of reduced data.
3. Application(s) specific assessments
All specific assessments performed are made in terms of compliance with the KPIs derived as described in reference document [D2].
4. Compliance with user requirements
The fulfillment of the derived KPIs (see [D2]) in terms of performance against the reference dataset has been demonstrated during periods of good data coverage.
Figures and Tables
Figure 4-1: Global daily mean ICDR UTH – ERA-Interim derived UTH differences, within +/-60 degrees latitude, for NOAA-18, 1/1/2016-20/10/2018.
Figure 4-2: Number of daily data matchups used to determine the data shown in Figure 4-1 month (2016-2020).
Figure 4-3: Global daily mean ICDR UTH – ERA Interim (to 31/12/2018), and ICDR UTH – ERA5 (from 1/1/2019) UTH differences, within +/-60 degrees latitude, for MetOp-A, 1/1/2016-31/8/2020 month (2016-2020).
Figure 4-4: Number of daily data matchups used to determine the data shown in Figure 4-3.
Figure 4-5: Global daily mean ICDR UTH – ERA Interim (to 31/12/2018), and ICDR UTH – ERA5 (from 1/1/2019)ERA-Interim derived UTH differences, within +/-60 degrees latitude, for MetOp-B,1/1/2016-31/8/2020 month (2016-2020)
Figure 4-6: Number of daily data matchups used to determine the data shown in Figure 4-5
Table 4-1: Showing the number of daily mean ICDR UTH – reanalysis derived UTH daily mean differences that fall within the defined KPI values (-0.47% to 0.55% (1/1/2016 to 31/12/2018), and 0.40% to 1.57% (1/1/2019 to 31/8/2020). For MetOp-A and MetOp-B the performance against each of the ERA-interim (2016-2018) and ERA-5 (2019-2020) derived KPI limits are shown separately.
Number of daily global (+/-60deg) mean UTH differences | Number of daily mean global (+/-60deg) UTH differences > \( \Big\{\begin{array}{ll} 0.55\% (1/1/2016 - 31/12/2018) \\ 1.57\% (1/1/2019 − 31/8/2020)\end{array} \) | Number of daily mean global (+/-60deg) UTH differences < \( \Big\{\begin{array}{ll} -0.47\% (1/1/2016 - 31/12/2018) \\ 0.40\% (1/1/2019 − 31/8/2020)\end{array} \) | Percentage of | KPI reanalysis baseline | |
NOAA-18 | 1023 | 48 (4.7%) | 21 (2.1%) | 954 (93.2%) | ERAinterim |
Metop-A \( \Big\{\begin{array}{ll} 2016 - 2018 \\ \textbf{2019 - 2020}\end{array} \) | 1095 | 32 (2.9%) | 3 (0.3%) | 1060 (96,8%) | ERAinterim |
Metop- B \( \Big\{\begin{array}{ll} 2016 - 2018 \\ \textbf{2019 - 2020}\end{array} \) | 1092 | 5 (0.5%) | 42 (3.8%) | 1045 (95.7%) | ERAinterim |
Table 4-2: showing the number of daily mean ICDR UTH – reanalysis derived UTH daily mean differences that fall within the TCDR 2.5 and 97.5 percentiles for each satellite. For MetOp-A and MetOp-B the performance against each of the ERA-interim (2016-2018) and ERA-5 (2019-2020) derived KPI limits are shown separately.
2.5 percentile KPI | 97.5 percentile KPI | Number of daily mean ICDR UTH diffs | Number of ICDR daily mean differences < TCDR 2.5 percentile | Number of ICDR daily mean differences > TCDR 97.5 percentile | Number of ICDR daily mean differences within TCDR 2.5 and 97.5 percentiles | KPI reanalysis baseline | |
NOAA-18 | -0.46 | 0.47 | 1023 | 22 (2.2%) | 83 (8.1%) | 918 (89.7%) | ERAinterim |
Metop-A \( \Big\{\begin{array}{ll} 2016 - 2018 \\ \textbf{2019 - 2020}\end{array} \) | -0.39 | 0.55 | 1095 | 8 (0.7%) | 32 (2.9%) | 1055 (96.3%) | ERAinterim |
Metop-B \( \Big\{\begin{array}{ll} 2016 - 2018 \\ \textbf{2019 - 2020}\end{array} \) | -0.47 | 0.46 | 1092 | 42 (3.8%) | 16 (1.5%) | 1034 (94.7%) | ERAinterim |
References
Hans, I.; Burgdorf, M.; John, V.O.; Mittaz, J.; Buehler, S.A. Noise performance of microwave humidity sounders over their lifetime. Atmos. Meas. Tech. 2017, 10, 4927–4945.
This document has been produced with funding by the European Union in the context of the Copernicus Climate Change Service (C3S), operated by the European Centre for Medium-Range Weather Forecasts on behalf on the European Union (Contribution Agreement signed on 22/07/2021).
All information in this document is provided "as is" and no guarantee of warranty is given that the information is fit for any particular purpose.
The users thereof use the information at their sole risk and liability. For the avoidance of all doubt, the European Commission and the European Centre for Medium-Range Weather Forecasts have no liability in respect of this document, which is merely representing the author's view.
