Contributors: Rory Gray (Met Office), Christoforos Tsamalis (Met Office), Roger Saunders (Met Office)
History of modifications
List of datasets covered by this document
Related documents
Acronyms
Scope of the document
This document is the Product Quality Assurance Document (PQAD), version 1.2, for the microwave upper tropospheric humidity (UTH) ICDR product. This product continues the time series started from the CM SAF dataset (1999-2015). The whole product validation methodology is described in detail in the CM SAF Validation Report [D1].
Executive Summary
This report concerns the validation of the CM SAF Upper Tropospheric Humidity (UTH) Interim Climate Data Record (ICDR) derived from the Microwave Humidity Sounder (MHS) on-board the series of NOAA satellites (18 (to Oct 2018) and 19) (https://www.wmo-sat.info/oscar/satellites/view/340, https://www.wmo-sat.info/oscar/satellites/view/341) and the MetOp satellites (A and B) (https://www.wmo-sat.info/oscar/satellites/view/306, https://www.wmo-sat.info/oscar/satellites/view/307), from January 2016 to May 2020. The ICDR has been validated against the equivalent UTH derived from ECMWF’s ERA-Interim (https://www.ecmwf.int/en/forecasts/datasets/reanalysis-datasets/era-interim) and ERA-5 (https://www.ecmwf.int/en/forecasts/datasets/reanalysis-datasets/era5) global atmospheric reanalyses in the same way as was done for the CM SAF equivalent product, details of which are provided in the CM SAF Validation Report [D1].
The ERA-Interim and ERA-5 equivalent UTH has been calculated using the NWP SAF radiance simulator, which constitutes an interface to the radiative transfer model of the NWP SAF RTTOV. RTTOV estimates the brightness temperature based on the ERA-Interim input, which is then converted to UTH using the same relationship as for the nadir observations of the microwave (MW) sounders. ERA-Interim offers the best available choice for validation given that humidity observations from many radiosonde types in the upper troposphere are affected by significant biases.
Key Performance Indicators (KPIs) were defined in terms of deviations of the UTH CDR from the ERA-Interim reanalysis reference data set, by the methods described in [D2, section 3]. Reference performance targets were set by determining the UTH limits of the 2.5 and 97.5 percentiles of the differences between the UTH TCDR and the ERA-Interim derived reference data set. For any of the satellites, no more than 2.5% of the TCDR data was found to have a UTH value more than 0.47% below the reference UTH, and no more than 2.5% of the TCDR UTH data was more than 0.55% higher than the reference UTH. These limits were taken as the KPI values.
Comparison of the ICDR dataset against the reference dataset using the defined KPIs confirms that, for MetOp-A,-B and NOAA-18, the ICDR reference dataset differs from the reference dataset in the same way that the TCDR dataset does. The relevant MHS channel on NOAA-19 (channel 3), has shown erratic behaviour since July 2009, and should not be used (Hans et al, 2017). Detailed results will be provided in the forthcoming ICDR Product Quality Assessment Report (PQAR) [D3], due for delivery at the end of November 2020.
1. Validated products
The validated product is the microwave upper tropospheric humidity (UTH), which is based on observations from the microwave (MW) sounder, the Microwave Humidity Sounder (MHS), on board the satellite platforms NOAA [18 and 19] and MetOp [A and B].
2. Description of validating datasets
Until July 2019, the validation of the C3S UTH product was conducted against the UTH calculated from the ERA-Interim reanalysis (https://www.ecmwf.int/en/forecasts/datasets/reanalysis-datasets/era-interim), as was done for the CM SAF equivalent product, described in detail in the CM SAF Validation Report [D1], and from August 2019 in the same manner against the UTH calculated from the ERA-5 (https://www.ecmwf.int/en/forecasts/datasets/reanalysis-datasets/era5) reanalysis.
3. Description of product validation methodology
The accuracy requirements applicable for this validation are the Key Performance Indicators (KPIs), defined in terms of deviations of the UTH ICDR from the ERA-Interim reanalysis reference data set, by the methods described in [D2, section 3.
The performance of the ICDR dataset against the ERA-Interim derived reference data set is compared with the performance of the already validated (as described in the CM SAF Validation Report [D1]) TCDRs against the reference dataset. Since August 2019, ERA-Interim has been unavailable. From August 2019, the performance is compared with the performance against ERA-5 obtained over the year 2018, the year that preceded ERA-Interim becoming unavailable. The same methodology was used to compare the ICDR against ERA-5 as was done to compare the ICDR against ERA-Interim, and is as described in the CM SAF Validation Report [D1].
Only data within ±60° latitude were considered for the performance assessment because very low water vapour loading in the upper troposphere outside those regions can make the UTH retrieval there unreliable. Upper and lower performance targets on the UTH difference between the TCDR UTH and reference dataset were chosen such that 95% of the UTH differences lie within the target range. 97.5% of the differences are above the lower target, and 2.5% of the differences are above the higher target. For any one satellite, no more than 2.5% of the TCDR data were found to have a UTH value more than 0.47% below the reference UTH, and a maximum of 2.5% of the TCDR UTH data were more than 0.55% higher than the reference UTH. These were taken as the KPI values. The number of values of the ICDR – ERA-Interim equivalent UTH differences falling within and without the performance targets was counted, and the relevant percentages of the total data points determined. Comparison of the ICDR dataset against the reference dataset using the defined KPIs confirms that, up to July 2019, the ICDR reference dataset differed from the reference dataset in the same way that the TCDR dataset does, and since August 2019 the ICDR performance against the reference dataset has remained similar to the performance throughout 2018. Detailed results will be provided in the forthcoming ICDR Product Quality Assessment Report (PQAR) [D3], due for delivery at the end of November 2020.
4. Summary of validation results
The main conclusion is that in terms of the defined key performance indicators, up to July 2019, when ERA-Interim stopped being available, the performance of the UTH ICDR product against the reference dataset is comparable with that of the UTH TCDR dataset against the ERA-Interim reanalysis reference. Since August 2019, the ICDR performance against the ERA-5 reanalysis reference dataset has been comparable with its performance over the previous year (2018) against that reference. Hence the key performance of the UTH ICDR is good (except for NOAA-19, whose relevant channel has had known issues since 2009 and should not be used (Hans et al, 2017)). A detailed summary of the validation results will be provided in the forthcoming C3S UTH ICDR Product Quality Assessment Report (PQAR) [D3], due for delivery 30/11/2020
Figures and Tables
Figures and tables will be provided in the forthcoming C3S UTH ICDR Product Quality Assessment Report (PQAR) (Water Vapour UTH_MW ICDR v1.1, Delivery D2.3.11-v1.1) [D3], due for delivery 30/11/2020.
References
Hans, I.; Burgdorf, M.; John, V.O.; Mittaz, J.; Buehler, S.A. Noise performance of microwave humidity sounders over their lifetime. Atmos. Meas. Tech. 2017, 10, 4927–4945.
For further references refer to the CM SAF ATBD [D1].
This document has been produced in the context of the Copernicus Climate Change Service (C3S).
The activities leading to these results have been contracted by the European Centre for Medium-Range Weather Forecasts, operator of C3S on behalf of the European Union (Delegation agreement signed on 11/11/2014). All information in this document is provided "as is" and no guarantee or warranty is given that the information is fit for any particular purpose.
The users thereof use the information at their sole risk and liability. For the avoidance of all doubt , the European Commission and the European Centre for Medium - Range Weather Forecasts have no liability in respect of this document, which is merely representing the author's view.
