Contributors: J. El Kassar (FUB), R. Preusker (FUB)
Issued by: Rene Preusker / Jan R. El Kassar
Date: 21/01/2021
Ref: C3S_312b_Lot1.2.3.6-v1.1_202011 _PQAR_ECV_WVP_TCWV_GV_TCDR
Official reference number service contract: 2019/C3S_312a_Lot1_FUB/SC1
History of modifications
List of datasets covered by this document
Related documents
Acronyms
List of tables
List of figures
General definitions
In the scope of the Copernicus Climate Change Service (C3S), a Thematic Climate Data Record (TCDR) has a fixed end point, whereas an Interim Climate Data Record (ICDR) is extended continuously. The TCWV_MERIS_SSMI_TCDR L3 product is a TCDR which consists of two satellite-based estimates of the Total Columnar Water Vapor (TCWV) of the atmosphere for the time period 2002-2012. The two estimates are derived by two different instruments over two types of surface. Over land, data from the Medium Resolution Imaging Spectrometer (MERIS) are used. Over water, the Special Sensor Microwave/Imager (SSM/I) is used.
All documents related to the TCWV_MERIS_SSMI_TCDR product are mostly limited to the processing of L2 MERIS data by the Freie Universität Berlin (FUB). SSM/I L2 data have been provided by the German Weather Service (DWD) and were not part of the processing at FUB and are thus not discussed extensively in this or any of the other supplementary documents. The validation results summary for SSMI/S L2 is included in this document, but extensive description of the validation method & procedure for L2 SSMI/S dataset can be found in [D2] .
Scope of the document
This Product Quality Assessment Report (PQAR) describes the product validation results of the Level 2 and Level 3 Total Column Water Vapor (TCWV) datasets from the Medium Resolution Imaging Spectrometer (MERIS) produced within C3S_312b Lot1 by the Freie Universität Berlin (FUB) and the SSMI/S dataset produced within the HOAPS by the Deutsche Wetterdienst (DWD). This Climate Data Record consists of only one merged product: the integrated water vapor content of the atmosphere. Information about the used metrics and validation methodology can be found in the related document: the Product Quality Assurance Document (PQAD) [D1].
Executive summary
The TCDR of Total Column Water Vapour (TCWV) from the Freie Universität Berlin (FUB) is an in-house product delivered to the Climate Data Store (CDS) of the Copernicus Climate Change Service (C3S).
The MERIS TCWV retrieval is based on the Cloud Aerosol and Water Vapor algorithm (CAWA) [Preusker et al., 2015] which was developed at the FUB within the Advanced Clouds, Aerosols and Water Vapour products for Sentinel 3/OLCI (CAWA) ESA Scientific Exploitation of Operational Missions Element Program (SEOM). The retrieval is applied to the MERIS instrument onboard the polar orbiting ENVISAT.
The SSMI TCWV is provided by DWD and is retrieved by SSM/I and SSMIS instruments onboard several polar orbiting DMSP satellites. For further information on the SSM/I SSMIS dataset we refer to Graw et al. (2017).
The MERIS TCWV and SSMI TCWV datasets are combined by the FUB team and comprises of 10 years (2002-2012) L3 monthly mean time series. The CDR provided here includes monthly means of TCWV on a regular global latitude-longitude grid, merged from MERIS (over land) and SSM/I (over ocean).
1. Product validation methodology
In this section, we will briefly summarize the product validation methodology. A more detailed explanation on this can be found in the Product Quality Assurance Document (PQAD) [D1].
1.1 Validated Products
We validated the TCWV product on two different processing levels. On the one hand we validated Level 2 MERIS TCWV on the original, irregular satellite projection. Collocation was achieved according to the criteria set in the PQAD and between the years 2009 to 2011, where sufficient numbers of reference stations of each network were available.
On the other hand we validated the Level 3 SSMI MERIS TCWV merged product in its high resolution (0.05°). Collocation was done for the full time period of the SSMI MERIS TCWV dataset. The detailed collocation criteria can be found in the PQAD.
1.2 Reference Datasets
Level 2 MERIS TCWV was compared against a multitude of networks with station numbers ranging from a few to hundreds all over the globe. The most important datasets are ARM (microwave radiometry), SUOMI GNSS (GPS), AERONET (sun photometry), GUAN (radiosondes) and GRUAN (radiosondes).
GUAN was used to validate against the finished (merged) Level 3 product, whereas ARM, SUOMI GNSS, AERONET and GRUAN were used to validate against the Level 2 MERIS TCWV product.
1.3 Key Performance Indicators (KPIs)
TCWV_GV product is compared against the validation benchmarks that have been set by CMSAF in the requirements review for HOAPS-4 TCWV (D2). They can be found in Table 1-1 and are divided into three categories: threshold (minimum usefulness), target (significant improvement compared to threshold) and optimal (no improvement necessary). The KPIs have been put together from requirements established in GCOS, WMO OSCAR and other literature.
Table 1-1: Overview of benchmarks to compare the validation results, taken from CMSAF Validation Report for HOAPS 4.0 [D2]
Category | Bias | cRMSD | Stability (bias trend) |
Threshold | 3 kg/m2 | 5 kg/m2 | 0.4 kg/m2/dec |
Target | 1.4 kg/m2 | 2 kg/m2 | 0.2 kg/m2/dec |
Optimal | 0.6 kg/m2 | 1 kg/m2 | 0.08 kg/m2/dec |
2. Validation results
2.1 Level 2 individual, instantaneous match-ups
Figures 2-1 a) to d) show the scatterplots between MERIS TCWV and collocated reference TCWV. Overall we see very good agreement between all datasets with high correlation, low bias and low cRMSD.
a) b)
c) d)
Figure 2-1: Each dot in the plots represents a collocation. The color represents the frequency of occurance, blue meaning low and red meaning high frequency.
Table 2-1: Summary of the Level 2 match-up validation
Reference Dataset | Bias [kg/m2] | cRMSD [kg/m2] | MAD [kg/m2] | Correlation | Offset and Slope |
AERONET | 2.24 | 2.56 | 1.92 | 0.98 | y = 0.49 + 1.12x |
ARM | -0.13 | 1.54 | 1.21 | 1.0 | y = -0.2 + 1.0x |
GNSS | 0.53 | 2.48 | 1.88 | 0.98 | y = 0.24 + 1.02x |
GRUAN | 0.7 | 1.8 | 1.23 | 0.99 | y = 0.35 + 1.02x |
We can observe very good agreement between ARM, GNSS and GRUAN concerning the bias. The bias in comparison with AERONET, however, is much higher. This discrepancy is most likely due to the dry bias of the AERONET TCWV dataset. This dry bias can amount to between 5 – 9% of TCWV in comparison with other means of TCWV retrieval such as microwave radiometry (ARM), GPS (SUOMI GNSS) or radiosondes (GRUAN) (see Ramírez et al., 2014).
Overall, all comparisons with reference datasets showed a cRMSD well within or close to the "target" category requirement. All biases except for the AERONET validation are well within the "target" category. This indicates a dataset of good quality with some room for improvement.
2.1.1 SSMI/S L2 validation results
For SSMI/S L2 dataset the validation results from the validation report of CMSAF HOAPS 4.0 [D2] are summarized in Table 2-2.
The comparison has been performed for two products, i.e. ERA-Interim and RSS-TMI (Remote Sensing System-TRMM Microwave Imager). According to Table 1-1, the optimal requirements for bias and cRMSD are met by SSMI/S L2 total column water vapour product toward comparison products. Further details on validation method can be found in the [D2] document.
Table 2-2: Results of Level 2 global mean intercomparison analysis for SSMI/S (HOAPS 4.0) taken from [D1].
Reference Dataset | Bias [kg/m2] | cRMSD [kg/m2] |
ERA-Interim (1988-2014) | 0.30 | 1.1 |
RSS-TMI (1998-2014) | -0.39 | 1.0 |
2.2 Level 3 Monthly observations
In this section we present the validation results for Level 3 monthly observations (as can be found in the TCWV_GV dataset).
Figure 2-2 shows the scatterplot of collocated GUAN monthly radiosonde data and MERIS, SSM/I and SSMIS TCWV grid boxes. Overall there were 33887 valid collocations (collocation criteria can be found in the PQAD [D1]).
Figure 2-2: Scatterplot of all collocated monthly mean TCWV values for the 960 GUAN stations. The color represents the frequency of occurrence, blue meaning low and red meaning high frequency. Overall there were 33887 valid collocations.
We observe a bias of -0.06 kg/m2, well within the “optimal” bias requirement. The data exhibit a cRMSD of 3.14 kg/m2 which shows that the data are well above threshold requirements but did meet the “target” requirement category.
There still is room for improvement, currently work is being done on the first revision of the TCWV_GV dataset, version 1.1. Nevertheless, the usefulness of the dataset v1.0 has been proven and we meet the requirements set by CMSAF.
Figure 2-3: Time-series of the cRMSD, MAD and Bias for the whole time period of the dataset for all 960 GUAN stations.
In figure 2-3 the time series of cRMSD, MAD and Bias is shown. The bias only exhibits a small trend of 0.18 kg/m2 per decade. The significance of this trend is low. The p-value against the null-hypothesis (that the bias trend is 0) is quite high at 0.29 (>0.05). Nevertheless, we observe a trend in the cRMSD of -0.8 kg/m2/dec. We have not looked into the possible causes for this trend.
The bias trend of 0.18 kg/m2/dec is well within the “target” requirements and suggests good usefulness of the dataset (See target requirement for stability of TCWV product in CMSAF Validation Report [D2], Table 6-6).
All results for the validation of the Level 3 monthly observations have been summarized in Table 2-3.
Table 2-3: Results of Level 3 monthly observation intercomparison analysis for different validation metrics between TCWV_GV and GUAN TCWV.
Reference Dataset | Bias [kg/m2] | cRMSD [kg/m2] | MAD [kg/m2] | Correlation | Offset and Slope | Decadal Bias Trend [kg/m2/dec] |
GUAN | -0.06 | 3.14 | 2.26 | 0.97 | y = -0.3 + 1.02x | 0.18 |
3. Application(s) specific assessments
N/A
4. Compliance with user requirements
The user requirements – as set by the KPIs taken from the CMSAF TCWV Validation Report for HOAPS 4.0 [D2]– were met. In all cases, the threshold requirements (minimum requirements) are achieved. For the instantaneous match-ups, the target requirements are met with a bias (excluding Aeronet, due to this networks dry bias) between -0.13 kg/m2 and 0.7 kg/m2 (target:1.4 kg/m2, optimal 0.6 kg/m2) and cRMSD between 1.54 kg/m2 and 2.56 kg/m2 (threshold: 5kg/m2, target: 2kg/m2).
For Level 3 monthly product showed to be well within the “threshold” category, if not even exceeding the “target” category as well and showed a decadal bias trend of 0.18 kg/m2/dec (target: 0.2kg/m2/dec).
These results suggest that all quality needs are met and the dataset is suited for all applications.
References
Graw K., A. Andersson, M. Schröder, K. Fennig, 2017: Algorithm Theoretical Baseline Document HOAPS version 4.0, DOI: 10.5676/EUM_SAF_CM/HOAPS/V002
Pérez-Ramírez, D., D. N. Whiteman, A. Smirnov, H. Lyamani, B. N. Holben, R. Pinker, M. Andrade, and L. Alados- Arboledas (2014), Evaluation of AERONET precipitable water vapor versus micro- wave radiometry, GPS, and radiosondes at ARM sites, J. Geophys. Res. Atmos., 119, 9596–9613, doi:10.1002/2014JD021730.
Preusker R., H. Diedrich and J. Fischer, 2015: Retrieval of Total Column Water Vapor from MERIS/OLCI and MODIS above Land- and Ocean Surfaces ATBD
