Contributors: Hans Gleisner (DMI), Nabiz Rahpoe (DWD), Jaqueline Drücke (DWD)
Issued by: SMHI/Karl-Göran Karlsson
Date: 06/10/2023
Ref: C3S2_D312a_Lot1.3.1.1-2022_TRGAD-WV_v1.1
Official reference number service contract: 2021/C3S2_312a_Lot1_DWD/SC1
History of modifications
Related documents
Acronyms
List of tables
List of figures
General definitions
Climate data records
Climate data compilations from observations are most often referred to as Climate Data Records (CDRs). However, the data records from satellites may consist of different types of quantities, from original radiances to derived products. Radiance data of climate quality are defined as Fundamental Climate Data records (FCDRs) while data records consisting of satellite-derived geophysical products are defined as Thematic Climate Data Records (TCDRs). In the ideal case the TCDRs should be derived by methods using FCDRs as input. However, if standards for the used radiances have not fulfilled the strict requirements for being classified as FCDRs, these radiances may be denoted Fundamental Data Records (FDRs). Notice that TCDRs can currently be based on either FCDRs or FDRs.
A special case of TCDRs are data records produced with short latency (e.g., shortly after the end of a month). These are called Interim Climate Data Records (ICDRs). The word Interim means that the data record has a higher uncertainty than the original TCDR since it has not been possible to use exactly the same input data as for the TCDR due to the short latency. Interim also means that a user may have to wait for the next edition of the TCDR to get a fully consistent and homogenous climate data record that includes data from the period with ICDR data. Normally ICDRs behave very similar to TCDRs but continuous monitoring of their quality is recommended.
Uncertainty parameters
The meaning of the terms uncertainty, accuracy and error is often difficult to interpret and may be treated differently in various referred documents. In this document, we adopt the following interpretation:
The accuracy, uncertainty or error of an estimated ECV is described by three differently contributing components:
The systematic error
The random error
The time-dependent error
The systematic error is commonly the mean error or the Bias. For non-Gaussian distributions of the error, the median or the mean absolute error can be a more useful quantity.
The random error is commonly the root-mean-squared deviation RMSD. Sometimes, the Bias is subtracted yielding the centred root-mean-squared deviation cRMSD. Notice that if the Bias is zero, the two mentioned quantities are equal and may be interpreted as the standard deviation of the error (often denoted standard error).
The time-dependent error is commonly the change in Bias over time (for ECVs over decades). We call this parameter stability.
More details on the estimation of these parameters are given in the Report on Updated KPIs (D3).
Testing the quality and consistency of TCDRs and ICDRs
This C3S project also deals with extensions of TCDRs, i.e. products derived from continued processing of the CDRs using the same methods and algorithms as originally used for TCDR production. We denote these CDRs Intermediate Climate Data Records, ICDRs. To evaluate the ICDR compliance with original TCDRs, a different approach in terms of defined requirements is followed. The ICDR is assessed on the basis of the TCDR distribution with respect to a reference validation source. After calculation of this distribution of differences, the ICDR is evaluated against the same reference and a binomial test is applied to verify that 95 % of the difference values lie within the upper and lower bounds of the TCDR difference distribution. The lower and upper bounds of the difference distribution is defined as the 2.5th and 97.5th percentiles of the difference distribution.
For further clarity, a binomial test is a way to test the statistical significance of deviations by referring to a theoretically expected distribution of observations. In this case, we use the theoretically expected distribution of observation differences which is estimated from the difference between TCDR results and corresponding results from a validation source. We now want to test if a corresponding but restricted, i.e., based on a shorter time series of ICDR results, difference distribution is similar in its shape to the original TCDR difference distribution. This can be tested by selecting one upper and one lower percentile in the original distribution (here, the 2.5th and 97.5th percentiles) and check how many samples will fall within or outside this restricted distribution if randomly extracting a number of samples. The resulting distribution of yes and no answers as a function of the number of samples can be described by the binomial distribution (see statistical standard literature for its definition). Consequently, this sample-based difference distribution from the ICDR can then be numerically compared with what could be expected from the reference distribution based on the TCDR. Based on this, one can judge whether the ICDR results are representative or not for the TCDR results. Deviations here would then indicate particular problems for the ICDR products (assuming that the character of reference observations does not change).
More details on the estimation of errors and uncertainty parameters are given in the Report on Updated KPIs (D3).
Product requirements
Depending on the data record producer, different product requirements may be applied and they are used to evaluate validation results. An often-used way to handle this is to define several levels of requirements where each level is linked to specific needs or priorities. A three-level approach like the following is rather common:
Requirement | Description |
|---|---|
Threshold requirement: | A product should at least fulfill this level to be considered |
Target requirement: | This is the main quality goal for a product. It should reach this level based on the current knowledge on what is reasonable to achieve. |
Optimal requirement: | This is a level where a product is considered to perform much better than expected given the current knowledge. |
Satellite product levels
Satellite-based products are often described as belonging to the following condensed description of processing levels, each one with different complexity and information content:
Level | Description |
|---|---|
Level-0: | Raw data coming directly from satellite sensors, often described as sensor counts. |
Level-1: | Data being enhanced with information on calibration and geolocation. Level-1a: Data with attached calibration and geolocation information Level-1b: Data with applied calibration and attached geolocation information Level-1c: Data with applied calibration and additional layers of geolocation, satellite viewing and solar angle information |
Level-2: | Derived geophysical variables at the same resolution and location as L1 source data. An often-used Level-2 variety is the following: Level-2b: Globally resampled images, two per day per satellite, describing both ascending (passing equator from south) and descending (passing equator from north) nodes. Resampling is based on the principle that the value for the pixel with the lowest satellite zenith angle is chosen in case two or several swaths are overlapping. |
Level-3: | Gridded data with results accumulated over time (e.g., monthly means). |
A more comprehensive definition of all processing levels is given here:
Radiation terms
Since satellite measurements are primarily about radiation measurements in different parts of the spectrum, some definitions or synonyms need to be explained. Roughly, the spectrum is usually sub-divided into one part where solar radiation dominates and one part where radiation emitted by the Earth and the atmosphere dominates.
The solar part is usually referred to as “visible (VIS)” radiation and covers approximately wavelengths smaller than 1 µm. Two sub-regions are often referred to, namely “ultraviolet (UV)” for radiation below approximately 0.38 µm, and “near-infrared (NIR)” for radiation between 0.78 µm and 1 µm (but sometimes claimed to continue up to 2.8 µm).
The part dominated by emitted radiation from the Earth is often referred to as “thermal” radiation. Common synonyms used are “infrared (IR)” or “terrestrial” radiation. Also here, we have several sub-regions defined. The “short-wave infrared (SWIR)” region is approximately defined by wavelengths between 1 µm and 2.5 µm. The “medium-wave infrared (MWIR)” region is approximately defined by wavelengths between 2.5 µm and 5 µm. The “long-wave” region, often simply referred to as just “infrared” to represent the bulk majority of radiation emitted by the Earth, defines radiation from approximately 5 µm up to about 1 mm. Radiation above 1 mm up to 10 cm is denoted “microwave (MW)” radiation.
Special terms
The term “AVHRR-heritage” is frequently used in the TRGAD documents. By this is meant spectral channels of other sensors than the AVHRR which show a close similarity (or heritage) to the AVHRR channels, i.e., having almost the same spectral characteristics.
A product is said to be “brokered” when an existing data record from an external source (i.e., not produced exclusively within this C3S project) is handled.
We can get a better idea of how accurate the final product values are by using the method of “error propagation”. It means that the retrieval method is capable of accounting for errors or uncertainties in the measurements or products used to derive the final product, e.g., radiances, input or ancillary data. In this way, the uncertainty of the final products can be estimated.
Radiation fluxes for radiation budget estimations are sometimes described as being “balanced”. It comes from the fact that instrument uncertainties for radiation budget measurements are often too high to be capable of providing accurate estimations of the net radiation fluxes at the top of atmosphere. Thus, balancing is a form of bias correction based on investigations of energy balance from other observations and model studies.
Calibration of radiances are sometimes described as based on “vicarious” methods. This indicates that there is no on-board mechanism on the satellite that provides the necessary calibration information. Consequently, parameters used in calibration equations have to be estimated retrospectively from historic data by use of additional references. For example, Earth surfaces which are considered to be invariant or stable are often used as reference targets for calibration of visible radiances.
An “OPeNDAP” server is an advanced software solution for remote data retrieval (see https://www.opendap.org/).
“Triple Collocation (TC)” is a large-scale validation technique by which error variances and data-truth correlation coefficients of three independent datasets can be estimated without a specific reference observation. For further details, see Stoffelen (1998).
Scope of the document
This document provides relevant information on requirements and gaps for retrievals of atmospheric water vapour contents based on data from three different sources:
1. Radio occultation measurements
2. Combined near infrared and microwave imagers.
The document is divided into three parts. Part 1 describes the products the present document refers to. Part 2 provides the target requirements for the products. Part 3 provides a past, present, and future gap analysis for the products and covers both gaps in the data availability and scientific gaps that could be addressed by further research activities (outside C3S).
Executive summary
The existence of water on Earth may be the single most important factor explaining why climate conditions are favourable for life and human existence on this planet. Atmospheric moisture is particularly involved in two vital processes: the water cycle and the atmospheric radiation. While precipitation and cloud products are dealt with in other TRGAD documents, we are here focussing on the water vapour in the atmosphere and methods and instruments for measuring water vapour content. Such measurements can be made in different ways and for the C3S project, two different measurement principles have been applied with the following two corresponding data records delivered:
- Method: Radio occultation measurements
- Data record: Water Vapour GRM TCDR v1.0 + ICDR v1.x
- Method: Measurements by combined near infrared and microwave imagers
- Data record: Water Vapour TCWV WV_cci/CM SAF (COMBI) TCDR v1.0
Each of the data records are described in this document together with their target requirements and a corresponding gap analysis section.
The first data record, i.e., the Water Vapour GRM TCDR product (v1.0, covering the period December 2006 – December 2016) and the associated ICDR product (starting in January 2017 with regular updates thereafter), are described together with target requirements and validation method descriptions. This product describes the vertical profile of specific humidity. The target requirement for the TCDR product consists of a 3 % mean error requirement on the monthly mean values. No stability requirement has been defined yet. The associated ICDR product is required to be consistent with the TCDR, which is formally checked through a binomial test.
An extensive description of past, current and future availability of radio occultation data and radio occultation measurements from low earth orbit (LEO) satellites is given. These measurements will continue well into the mid-2030s and even longer. Currently achieved retrievals of humidity profiles are associated with a small negative bias in the lower troposphere. Some positive biases are seen for higher elevations which mainly are explained by the use of background temperatures from reanalysis fields (ERA-Interim or ERA5) near the tropopause. Atmospheric sounding based on Global Navigation Satellite System Radio Occultation (GNSS-RO) measurements is still a field of active research. Further improvements are partly linked to the availability of improved background data from new reanalysis efforts. Also, development of methods for better bias corrections is ongoing.
The second data record, TCWV WV_cci/CM SAF (COMBI) TCDR v1.0, is presented together with target requirements. This product describes the total column water vapour in the atmosphere. The target requirements (based on comparisons with AIRS data as a reference dataset) are set to 2.5 % for the bias and 0.5 %/decade for the stability.
The past, present and future availability and capabilities of both the microwave imager SSM/I SSMIS and the water vapor sensitive NIR imager MERIS and successor OLCI are shown. The TCDR is running from 2002 to 2017. OLCI is in orbit since 2016. The gap between 2012 and 2016 is bridged with MODIS onboard the Terra satellite.
The COMBI product provides improved NIR-based retrievals compared to previous products based exclusively on NIR data from MERIS on the ENVISAT satellite. This concerns both the used retrieval method and the spatial coverage, with additional NIR data now provided by the MODIS and OLCI sensors. NIR-based products provide valid retrievals over land, under clear-sky conditions and during day time as a complement to microwave-based retrievals which are only applicable over ocean surfaces. The latter retrieval of TCWV over ocean is based on a 1D-Var retrieval scheme applied to microwave imager observations. This retrieval was originally developed by the NWP SAF and is operated by CM SAF to generate the HOAPS dataset.
We repeat that microwave-based results from TCWV retrievals from HOAPS over ice-free ocean complement NIR-based retrievals over land, sea-ice and coasts from MERIS, MODIS and OLCI. The microwave-based retrievals are optimised for dark surfaces and does not work well over bright sea-ice.
A future outlook for the use of Sentinels (apart from OLCI onboard Sentinel 3) as well as systems such as IASI and MTG and their benefits to a future TCWV dataset, is outlined.
1. Product description
The existence of water on Earth is maybe the single most important factor explaining why climate conditions are favourable for life and human existence on this planet. Atmospheric moisture is particularly involved in two vital processes: The water cycle and the atmospheric radiation. While precipitation and cloud products are dealt with in other TRGAD documents, we are here focussing on the water vapour in the atmosphere and methods and instruments for measuring water vapour content. Such measurements can be made in different ways and for the C3S project, the following two different measurement principles have been applied:
- Radio occultation measurements
- Combined near infrared and microwave imagers.
Measurements of atmospheric moisture are not trivial which explains why several measurement principles are applied. For example, measurements can be disturbed by the fact that both clouds and precipitation may be present in the sensors field-of-view (FOV). This problem can at least partly be solved by measuring at microwave wavelengths where the influence of clouds and precipitation is much lower. On the other hand, microwave measurements have often coarser FOV resolution than infrared measurements. Thus, a combination of infrared and microwave measurements can be favourable.
Conditions in the upper troposphere are especially interesting since changes here (where the moisture contents generally are low) could quickly affect radiation conditions (especially the atmospheric greenhouse effect). The knowledge of upper atmospheric humidity is not very well established since measurements are often dominated by contributions from the lower parts of the atmosphere. Because of this, measurements made from radio occultations are especially interesting since these are limb measurements, i.e., looking from the side and not directly downwards in the atmosphere. This yields more accurate results at higher levels where clouds and precipitation cannot obscure measurements to the same extent as for the lower troposphere.
1. 1.1 Water Vapour GRM TCDR v1.0 + ICDR v1.x
The Water Vapour GRM TCDR product consists of monthly mean values of tropospheric specific humidity covering the time period December 2006 to December 2016. The data product is provided on a monthly mean latitude-altitude grid (horizontal resolution 5˚x5° and vertical resolution 200 meters), with a global coverage and extending from the surface up to 12 km. Notice that the “GRM” notation is not an acronym but an internal product code applied by the ROM SAF (the latter acronym explained in the next sentence).
The monthly-mean gridded data are generated by the EUMETSAT radio occultation meteorology satellite application facility (ROM SAF) from measurements made by the GRAS Radio Occultation (RO) instruments onboard the Metop polar-orbiting satellites. From observed atmospheric refractivity profiles, and background information from reanalysis fields (currently, ERA-Interim), near-vertical tropospheric humidity profiles are retrieved through a 1D-Var approach. The retrieved humidity profiles are averaged into monthly means at the global latitude-altitude grid. Compared to other satellite observational techniques, RO data have high vertical resolution, and the observations are not affected by clouds or the underlying surface (e.g., no land-sea differences).
The ICDR product extends the TCDR time series after 2016. The ICDR data record starts in January 2017 and is regularly updated. At the time of writing this report, ICDR data up to August 2022 are available in the Climate Data Store. The data are generated from the same instruments as the corresponding TCDR data, using the same algorithms applied to data available at the time of the ICDR processing and has the same general characteristics (resolution, coverage, etc.) as the corresponding TCDR product. The ICDR processing initially used background data from ERA-Interim, i.e., the same as the TCDR. However, a switch of background data to ERA5 was done in August 2019 as the ERA-Interim processing was terminated at ECMWF.
2. 1.2 Water Vapour TCWV WV_cci/CM SAF (COMBI) TCDR v1.0
The Total Column Water Vapour (TCWV) WV_cci/CM SAF (COMBI) product is a daily and monthly mean product on a regular grid, available at both low resolution (0.5° x 0.5°) and high resolution (0.05° x 0.05°). It covers the period July 2002 to December2017 and combines near-infrared (NIR)-based retrievals over land, coasts and sea-ice and microwave imager based over open ocean (see Figure 1-1 for overview). Over open ocean, the Special Sensor Microwave Imager (SSM/I) is used. Over land, coastal and inland waters, the Medium Resolution Imaging Spectrometer (MERIS), the Ocean and Land Colour Instrument (OLCI) and Moderate Resolution Imaging Spectrometer (MODIS) is used. MERIS is an instrument on a single satellite (ENVISAT) that flew for 10 years (2002-2012). SSM/I (and its successor SSMIS) flew on several different platforms in the Defense Meteorological Satellite Program (DMSP). The used MODIS instrument is aboard the TERRA satellites. OLCI is an instrument on the Sentinel-3a/b satellites, which were launched in 2016.
Figure 1-1: Instruments used for TCWV WV_cci/CM SAF (COMBI) product. MERIS, MODIS, and OLCI provide data for land and sea ice, while HOAPS provides data for the global ice-free ocean.
SSM/I is a well-established instrument for TCWV retrieval over open ocean surfaces and provides a long time series and global coverage at a reasonable resolution of approximately 25 km.
SSM/I TCWV data has been processed with the Hamburg Ocean and Atmosphere Parameters and Fluxes from Satellite (HOAPS) algorithm version 4.0 and was provided to Freie Universität Berlin (FUB) by DWD. SSM/I cannot retrieve water vapor over land surfaces and over areas with strong precipitation.
MERIS TCWV is retrieved with the differential absorption technique at the water vapour absorption peak between 890 nm and 1000 nm with its centre at 940 nm. TCWV is estimated from the ratio between the radiance at a band inside the absorption peak (absorption channel 15, and the radiance at a band more outside of the peak (window channel)). The retrieval process is described in more detail in reference document [D6]. MERIS cannot retrieve TCWV over clouds or areas with an uncertain surface type. MERIS TCWV has a resolution of ca. 1 km. MERIS only retrieves TCWV on the descending node with an equator crossing time (ECT) at 10:00 AM. Thus, SSM/I data were filtered for the descending nodes and with ECTs in the morning (5:00 to 9:00).
In the case of MERIS grid boxes this is further defined as the monthly average of morning clear sky TCWV. TCWV fields are achieved by calculating the error-weighted average over all daily composites of SSM/I and MERIS. Since the resolution of SSM/I is not fine enough for the high resolution, SSM/I grid boxes at 0.5° were up-sampled 10 times with the nearest neighbour method to achieve a resolution of 0.05°.
The MODIS instruments provide high radiometric sensitivity (12 bit) in 36 spectral bands ranging in wavelength from 0.4 μm to 14.4 μm. A ±55-degree scanning pattern at a sun-synchronous polar orbit of 705 km achieves a 2,330-km swath and provides global coverage every one to two days. The MODIS instrument is flying on board the Terra (10:30 equator crossing time) and on board Aqua (13:30 equator crossing time). For the water vapour retrieval, the MODIS spectral bands of 858.5 nm (250m spatial resolution) and 1240 nm (1 km spatial resolution) are used as reference and the three channels between 905 nm and 940 nm for the water vapour absorption [D6].
OLCI bands are optimised to measure ocean colour over the open ocean and coastal zones. A new channel at 1020 nm has been included to improve atmospheric and aerosol correction capabilities, additional channels in the O2 A-band spectral region are included for improved cloud top pressure (height) and water vapour retrieval, and a channel at 673 nm has been added for improved chlorophyll fluorescence measurements. For the water vapour retrieval, the OLCI spectral bands of 885 nm, 900 nm and 1020 nm are used as reference channels and two channels at 900 nm and 940 nm are used as absorption channels [D6].
The NIR data over land were combined with TCWV data from EUMETSAT CM SAF Hamburg Ocean Atmosphere Parameters and Fluxes from Satellite Data (HOAPS) over open ocean. This is the total column water vapour COMBI product.
2. User Requirements
This section describes the requirements which have been set to be achieved by the described products. Requirements can be set at different levels (as explained in the section with General definitions) but here we will focus on what is called the Target Requirements. These requirements define the main goals for data producers which have to be fulfilled by their products. Requirements are specified by the use of various accuracy parameters which are also listed in the section with General definitions. Observe that for brokered products the target requirements are set to the achieved validation results since these products are not developed and tested within the C3S project.
Concerning products to be used in climate monitoring, requirements for what should be achievable through Earth Observation systems are generally defined by the World Meteorological Organisation (WMO) Global Climate Observation System (GCOS) expert panel. However, these requirements are generally oriented towards the capability and resolution of climate models with a rather course spatial resolution while many products listed here are focusing more on the monitoring of local and regional scale conditions. Also, they are often not attainable using existing or historical observing systems. Thus, GCOS requirements are not always identical to the requirements listed here since also other user groups than the climate modelling community have contributed in setting the requirements. However, the relation to GCOS requirements are discussed below for each individual product.
3. 2.1 Water Vapour GRM TCDR v1.0 + v1.x
3.1. 2.1.1 Summary of target requirements (KPIs)
The target requirements, as specified by the KPIs (see Table 2-1), consist of a 3% mean error requirement on the monthly mean values within the fundamental latitude-altitude bins (5˚ by 200 m). The requirement is tested by requiring that at least 60% of the observed bin values deviate by less than 3% from ERA-Interim. For each month, this test is applied to three broad latitude zones: tropics (30˚S–30˚N), mid-latitudes (30˚N–60˚N and 30˚S–60˚S), and polar latitudes (60˚N–90˚N and 60˚S–90˚S), and two tropospheric altitude intervals (0-8 km and 8-12 km), altogether six latitude-height regions (Figure 2-1).
Table 2‑1: Target requirements for the GRM specific humidity product.
Variable | KPI: accuracy (Bias) | KPI: decadal stability |
Spec. Hum. | 3 % | - |
The consistency between the ICDR version of the Water Vapour GRM specific humidity product and the corresponding TCDR product is checked by a test designed to detect certain type of differences between the ICDR and the TCDR [D3]. The relative differences between the monthly mean observed data and a reference data set are computed on a global latitude-height grid, for both the ICDR and the TCDR. These relative differences are globally averaged (properly area weighted) and vertically averaged (in 0-4 km, 4-8 km, and 8-12 km layers). For each vertical layer, we find the 2.5% and 97.5% percentiles of the TCDR differences. These percentiles are used in a binomial test to check whether the corresponding ICDR differences are consistent with the TCDR differences. Table 2-2 shows the actual values used for the limit percentiles in the binomial test.
Figure 2‑1: The accuracy requirements applicable to the Water Vapour THP products are checked for each month and within six broad latitude-altitude regions here referred to as regions I to VI.
Table 2‑2 KPIs for the ICDR of the GRM specific humidity product
Variable | KPI: lower percentile (2.5 %) | KPI: higher percentile (97.5 %) |
Spec. Hum. | 0-4 km: -2.00 % 4-8 km: -1.04 % 8-12 km: -0.66 % | 0-4 km: -0.93 % 4-8 km: +0.82 % 8-12 km: +2.13 % |
The ICDR used ERA-Interim as a reference data set initially but from August 2019 and onward ERA5 is the reference.
3.2. 2.1.2 Discussion of requirements with respect to GCOS and other requirements
The GCOS 2016 Implementation Plan states a 5% measurement uncertainty (mean error) requirement on “tropospheric profiles of water vapour”. Assuming that this requirement applies to monthly mean values, the accuracy requirement used for the C3S monthly-mean gridded THP TCDR humidity profiles is well in line with the GCOS requirements (Table 2-3).
Table 2‑3: Comparison of the GRM TCDR target requirements for specific humidity with the GCOS target requirement.
Requirement | GCOS (Target) | GRM TCDR |
Mean error | < 5% | <3% |
3.3. 2.1.3 Data format and content issues
This dataset provides monthly mean values of specific humidity in the troposphere, below an altitude of 12 km. The data are provided as monthly netCDF files containing the specific humidity on a global latitude-altitude grid as well as the monthly variability, and associated variables such as data numbers, sampling errors, and an estimate of the fraction of a priori (background model) information in the humidity data.
However, unlike the TCDR, which is generated in a reprocessing activity, the ICDR is regularly updated, currently on a quarterly basis. ICDR data up to August 2022 are currently available in the Climate Data Store.
4. 2.2 Water Vapour TCWV WV_cci/CM SAF (COMBI) TCDR v1.0
4.1. 2.2.1 Summary of target requirements (KPI)
The global TCWV WV_cci/CM SAF_(COMBI) TCDR product (covering the time period 2002-2017) is compared to different reference datasets: AIRS, ERA5, C3S and Gome Evolution. The results for the accuracy and decadal stability are listed in Table 2-4. More results can be found in the Validation Report [D8].
Table 2‑4: Achieved validation results) for the Water Vapour TCWV WV_cci/CM SAF (COMBI) TCDR v1.0. Results derived from comparisons with AIRS data are chosen as the target accuracies and are marked in bold text.
Variable | Reference dataset | KPI: accuracy (Bias) | KPI: decadal stability |
TCWV | AIRS ERA5 C3S GOME Evl | 2.5±0.6 % 0.5±0.5 % 0.3±0.4 % 3.1±0.9 % | 0.5±0.4 % 0.7±0.2 % - - |
4.2. 2.3.1 Discussion of requirements with respect to GCOS and other requirements
The validation relies on analysing results from comparisons to various satellite, reanalysis, ground-based and in situ data records. Global comparisons were carried out against AIRS, ERA5 and GOME Evolution (see Table 2-5). It was demonstrated that the combined global TCWV data record frequently meet the GCOS target requirements. Relative to AIRS (chosen as the basis for target requirements) mean bias and stability are 0.47 kg/m2 (2.5%) and 0.08 kg/m2/decade (0.5%), respectively (Section 7 in [D2]). Separate results for land and ocean only coverages and more details are given in the Validation Report [D2].
Table 2‑5: Global comparison of the Water Vapour TCWV WV_cci/CM SAF (COMBI) TCDR v1.0 validation results (where AIRS results, chosen as the target requirement, are marked in bold) with the GCOS target requirement. Marked in green is where the mean error and stability values fulfills the GCOS requirements.
Requirement | GCOS (Target) | Reference dataset | TCWV WV_cci/CM SAF (COMBI) TCDR |
Spatial resolution | 25 km | - | 5 km |
Temporal resolution | 4h | - | Daily |
Accuracy (mean error) | < 2 % | AIRS | 2.5±0.6 % |
ERA5 | 0.5±0.5 % | ||
C3S | 0.3±0.4 % | ||
GOME Evl | 3.1±0.9 % | ||
Stability | 0.3 % | AIRS | 0.5±0.4 % |
ERA5 | 0.7±0.2 % |
It is noted that the quality over inland water bodies, coastal areas and sea-ice is lower. Depending on the user application, it might be prudent to filter the data accordingly. It was also observed that the transition between MODIS and OLCI based TCWV over land between March and April 2016 is associated with a break point when compared to AIRS and ERA5. Thus, the OLCI period from April 2016 onwards should be excluded from climate change analysis. The NIR based TCWV data over land exhibits a high stability when OLCI data is removed and only clear-sky data is considered. Over ocean a small break point was observed when compared to the merged microwave data record from REMSS. However, the stability is still better than the target product requirement, though not significantly [D2].
GCOS also states that the frequency of TCWV observations should be 4 h, with a spatial resolution of 25 km. While the spatial resolution is barely achievable with a microwave imager such as SSM/I (footprint approx. 50 km with 25km spot spacing), MERIS, MODIS and OLCI (and other NIR imagers with similar band configurations) have resolutions of approx. 1 km or higher. Thus, the required resolution is achievable.
However, the temporal resolution of 4 h is not achievable in the current setup of polar-orbiting satellites. Even so the daily composites of MERIS consist of single measurements (at the poles multiple measurements) per day. SSM/I and SSMIS could provide a higher number of values per day. The achieved temporal resolution of the TCWV product is daily means, which is still reasonable.
4.3. 2.3.2 Data format and content issues
This dataset provides daily and monthly mean values of global total column water vapour with a spatial resolution of 0.5° and 0.05° respectively. The water vapour of the atmosphere is vertically integrated over the full column and given in units of kg/m2. The data are provided as daily and monthly netCDF files for each of the spatial resolutions containing the atmosphere water vapour content on a global regular latitude-longitude grid and related variables such as means, standard deviations, the number of observations and days, surface type flag, and the average and random retrieval uncertainty of the total column water vapour.
3. Gap Analysis
5. 3.1 Description of past, current and future satellite coverage
5.1. 3.1.1 Water Vapour GRM TCDR v1.0 + ICDR v1.x
The Water Vapour GRM TCDR is based on soundings of the Earth’s atmosphere by RO instruments onboard satellites in low-Earth orbit. The RO technique exploits measurements of the amplitude and carrier phase of signals from Global Navigation Satellite System (GNSS) satellites, as they set or rise at the Earth’s limb. Most RO sounding instruments up to date have been based on signals from the Global Positioning System (GPS), but there are also instruments that can utilize signals from the European Galileo system, the Chinese Beidou system, and the Japanese Quasi-Zenith Satellite System (QZSS). RO measurements made by scientific satellite missions have been available since 2001, while operational or quasi-operational RO measurements started in 2006.
The most Important of the operational missions are:
COSMIC-1: Six-satellite constellation launched in 2006. It is presently (2022) beyond its end of life, with only a single satellite remaining operational. Precessing satellite orbits ensure full coverage of local time over a time period of several months.
EPS: The EUMETSAT Metop satellites were launched in 2006, 2012, and 2018. The satellite orbits are Sun-synchronous with ascending node 09:30.
- FY-3C: Chinese satellite launched in 2013. The first in a planned series of operational RO missions. Sun-synchronous satellite orbit with ascending node 22:00.
- COSMIC-2: Six-satellite constellation launched in 2019. Precessing low-inclination satellite orbits, with coverage of low- and mid-latitudes only.
- EPS-SG: The EUMETSAT Metop-SG satellites are planned for a first launch in 2023 (two satellites). Follow-on launches will ensure continuity into the late 2030s.
- Sentinel-6: The Sentinel-6 mission (also referred to as Jason-CS) consists of two satellites, one launched in 2020 and the other scheduled for launch in 2025.
- Precessing medium-inclination satellite orbits. RO instrument with heritage from the COSMIC-2 mission.
These operational missions, run by governmental or inter-governmental agencies, ensure the availability of RO data well into the mid-2030s and even longer. We can also foresee a number of RO research missions as well as commercial missions that will provide data in addition to the operational missions. The data numbers will vary considerably during the next decade, being temporarily boosted by the COSMIC-2 mission. The future of the commercial missions, and of the technical and commercial developments of nano-satellite systems, is currently uncertain. However, long-lasting data gaps are very unlikely during the next 20 years.
Despite the availability of RO data, there may be limitations related to coverage of local solar time and, hence, the diurnal cycle. Operational, meteorological satellites are preferentially placed in Sun-synchronous orbits, with limited local-time coverage. The operational satellite missions are important as “backbone”, but missions with other type of orbits provide valuable information on the diurnal cycle, which is important when generating RO-based climatologies.
The Water Vapour GRM ICDR products are based on data from the same instruments and satellites as the corresponding TCDR.
5.2. 3.1.2 Water Vapour TCWV WV_cci/CM SAF (COMBI) TCDR v1.0
The microwave imagers (SSM/I and since 2005 the successor SSMIS) onboard the DMSP satellites are providing TCWV over open ocean since 1987. The latest addition to the satellite family DMSP F19 stopped providing useful data in 2016. EUMETSAT is providing continuity of the SSMIS time series with the envisioned launch of MWI in 2025.
The MERIS instrument only flew on the ENVISAT satellite from 2002 until the mission end in 2012.
The successor of MERIS, the Ocean and Land Colour Instrument (OLCI), is in orbit since 2016 and 2018 onboard Sentinel 3-A and –B, respectively. These imagers are used to extend the time series. They have the same ECT and band configuration like MERIS. The gap between 2012 and 2016 is bridged by using the Moderate Resolution Imaging Spectrometer (MODIS) onboard the Terra satellite. MODIS Terra has a similar band configuration and ECT of 10:30 AM UTC compared to MERIS. Diedrich et al. (2014) have shown that the approach used for MERIS TCWV is working with MODIS as well.
The continuity of the OLCI instrument is assured by COPERNICUS with launches of Sentinel 3-C and –D in 2023 and 2025.
The next generation of EUMETSATs geostationary satellites, Meteosat Third Generation (MTG) and the successor of the Advanced Very High Resolution Radiometer (AVHRR) series – METImage – will be operational from 2023/2024/2025. Both provide a single band in the H2O absorption band between 890 nm and 1000 nm. Thus, TCWV WV_cci/CM SAF (COMBI) processing is directly applicable to these instruments as well.
6. 3.2 Development of processing algorithms
6.1. 3.2.1 Water Vapour GRM TCDR v1.0 + ICDR v1.x
Atmospheric sounding using GNSS-RO measurements is a field of active research. The measurement technique itself, and the processing methods employed, are still in a state of development. The tropospheric humidity retrieval starts with the phase and amplitude of GNSS radio signals as measured by a GNSS receiver onboard a satellite. The retrieval of humidity profiles includes two steps: a) the bending (refraction) angles are computed using a wave optics method, followed by retrieval of the atmospheric refractivity through an Abel integral [D4], and b) vertical profiles of specific humidity are calculated by combining the observed refractivity with information from a priori (background) data taken from an atmospheric model [D5]. The combination of observations and background is done by means of a 1D-Variational (1D-Var) optimization algorithm. The retrieval of humidity profiles is followed by averaging into sampling-error corrected gridded monthly mean data [D1].
The observed refractivity profiles often tend to be negatively biased in the moist lower troposphere. This leads to biases in the humidity climatologies. Technically, the problem is related to challenges in tracking the signals from the GNSS satellites under turbulent conditions in the lower troposphere, as well as limitations in the retrieval algorithms under such conditions. A related phenomenon known to introduce errors into the observed refractivity profiles, leading to errors in the humidity retrievals, is ducting (also known as super-refraction) associated with the planetary boundary layer. There are ongoing development activities aimed at improved identification of ducting situations, and better methods for reducing the impacts of negative refractivity bias within the planetary boundary layer.
As mentioned above, the humidity retrievals require a priori (background) data. The retrievals rely on the availability of background data with accurate error characteristics and a vertical resolution that match the RO observations. The background data is most commonly taken from an atmospheric model, e.g., a numerical weather prediction model or a reanalysis. In the current ROM SAF processing for humidity TCDRs and ICDRs, ECMWF reanalyses are used as background. TCDR v1.0 and ICDR v1.0 use ERA-Interim, while ERA5 is used for ICDR v1.1 starting in August 2019. The next versions of TCDR and ICDR will only use ERA-5.
The RO retrieval methods are still evolving. For the generation of TCDRs, where temporal stability has high priority, filtering out of bias shifts and long-term variability in the background data is being investigated. Also, other types of development activities aimed at improving the temporal stability are foreseen.
The Water Vapour GRM ICDR data products are based on the same processing algorithms as the corresponding TCDR, and any development activities apply equally to both.
6.2. 3.2.2 Water Vapour TCWV WV_cci/CM SAF (COMBI) TCDR v1.0
Currently there are no plans to change the core of the algorithm. Further validation of the algorithm with ground truth will be used to insert some correction of possible systematic offsets. The only part of the algorithm that is further refined is the retrieval of TCWV over water since it is the most difficult surface to retrieve TCWV with the differential absorption technique. Better aerosol retrievals will improve the TCWV product over water substantially and decrease uncertainty.
7. 3.3 Methods for estimating uncertainties
7.1. 3.3.1 Water Vapour GRM TCDR v1.0 + ICDR v1.x
The Water Vapour GRM TCDR product consists of monthly means of tropospheric specific humidity profiles, generated by a relatively straight-forward binning-and-average technique. The error of the monthly mean is assumed to be caused by two effects. First, each measurement has a random measurement error associated with it. This error is described in terms of a statistical uncertainty. Secondly, the finite number of measurements is not able to fully account for all variability within the latitude bin and time interval, resulting in a sampling error. Unlike the measurement errors, it is possible to estimate the actually realized sampling errors in the monthly means. This allows us to make a correction of the observed means, leaving a residual sampling error. The residual sampling error is assumed to be random, and is described in terms of a statistical uncertainty.
The measurement uncertainty of the individual humidity profiles is obtained from the formal errors resulting from the 1D-Var retrievals [D5]. The measurement uncertainty of the mean is then obtained under the assumption that the humidity profiles in a latitude-month bin have uncorrelated errors, and also taking the weighting applied to the profiles into account. This is described in detail in [D1].
The sampling errors are estimated by sub-sampling an atmospheric model (currently, the ERA-Interim reanalysis) at the observed times and locations [D1]. Based on these estimates, we do a sampling-error correction by subtracting the estimated sampling errors from the observed means, leaving only residual sampling errors. The uncertainties related to the residual sampling errors are estimated as a certain fraction of the original estimated sampling errors.
The measurement uncertainties and the uncertainties due to the residual sampling errors are finally combined to a total uncertainty for the monthly mean [D1]. The two components of uncertainties in the monthly means are provided together with the gridded monthly-mean data products. In principle, there is also a structural uncertainty due to algorithmic choices and underlying processing assumptions, but these are currently not provided together with the data. Neither is there any information on error covariances (error correlations) in the gridded data, which may be requested by users in the future. However, it should be noted that very few ECVs are currently provided with that type of detailed error (uncertainty) descriptions.
The same methods for estimating the uncertainties are used for the Water Vapour GRM ICDRs as for the corresponding TCDRs.
7.2. 3.3.2 Water Vapour TCWV WV_cci/CM SAF (COMBI) TCDR v1.0
TCWV WV_cci/CM SAF (COMBI) is based on a 1D-Var retrieval, both in the case of the SSM/I (and SSMIS) TCWV from the HOAPS algorithm and NIR-based (MERIS/MODIS/OLCI) TCWV algorithm. 1D-Var retrievals are characterized by a linear error propagation which considers instrument uncertainties and characteristics (i.e. SNR) as well as uncertainties in auxiliary data (i.e. a priori knowledge) and atmospheric and surface conditions (i.e. aerosol load, surface brightness). The error propagation yields reliable error estimates for each retrieved value.
8. 3.4 Opportunities to improve quality and fitness-for-purpose of the CDRs
8.1. 3.4.1 Water Vapour GRM TCDR v1.0 + ICDR v1.x
Section 3.2.1 indicated some opportunities to improve the processing of RO measurements into profiles of tropospheric humidity. The aim of these improvements would be to reduce lower-tropospheric biases, but also to gain a better long-term temporal stability of the humidity time series. Any improvements of the humidity profile data lead to corresponding improvements of the monthly-mean humidity climatologies.
Reprocessing activities planned for by the EUMETSAT ROM SAF will utilize background data from new, modern reanalyses that are currently made available. The new background data will include improved error characterization. These developments are expected to lead to smaller systematic biases, better temporal stability, and more accurate description of the tropopause in future CDRs. In addition, filtering out of longer-term variability in the background data could be considered as a means to further improve the temporal stability.
Another line of development that can be foreseen as RO data numbers increase is the use of methods for global mapping of the RO data onto a latitude-longitude grid, instead of the currently available zonal latitude grid. Such methods exist but are currently not widely used.
The opportunities to improve the CDRs apply equally to both the TCDRs and the ICDRs.
8.2. 3.4.2 Water Vapour TCWV WV_cci/CM SAF (COMBI) TCDR v1.0
TCWV from NIR imagers is an established product. However, there are many ways to improve the quality and applicability of this data set.
We still observe stratification of systematic errors due to specific environmental conditions: e.g. high, lifted aerosols above dark surfaces. The quantitative validation of the uncertainties with ground truth would certainly be beneficial to the final product’s quality. Also, the TCWV retrieval over bright sea ice is challenging.
Additionally, further investigations are needed to quantify place and time dependent clear sky biases of NIR imagers.
With the inclusion of other platforms, especially future geostationary satellites, the TCWV from NIR will reach a new and higher level (see section 3.6.2).
9. 3.5 Scientific Research needs
9.1. 3.5.1 Water Vapour GRM TCDR v1.0 + ICDR v1.x
The tropospheric humidity retrievals depend on the availability of background data that are free from systematic biases. The uncertainties of the background data are assumed to be purely statistical with zero mean. In practice, a state-of-the-art reanalysis model is used as background in the 1D-Var retrievals. However, even the newest reanalysis models are not totally free of biases. One way to deal with this would be to develop and apply bias correction to the model data, something that would require detailed investigations into best practices, and how to ensure that artificial trends are not introduced in the retrieved data products.
Characterization of the uncertainties of humidity profiles as well as of gridded monthly-mean humidity data is essential for many applications. To obtain a reliable and accurate description of the uncertainties requires cross-comparisons with other observational data types, as well as a theoretical understanding of the error sources involved.
9.2. 3.5.2 Water Vapour TCWV WV_cci/CM SAF (COMBI) TCDR v2.0
Scientific research needs overlap with points made in the previous section 3.4.2. However, one big aspect is discontinuities between bright land and dark water surfaces. These could be tackled by combining NIR and thermal IR retrievals.
A much more fundamental aspect would be the investigation of smaller scale water vapor structures present at the onset of cloud formation processes. We can observe these to a limited extent but particularly Meteosat Third Generation (MTG) will deliver highly valuable data and insights.
10. 3.6 Opportunities from exploiting the Sentinels and any other relevant satellite
10.1. 3.6.1 Water Vapour GRM TCDR v1.0 + ICDR v1.x
The Sentinel-6 mission (also referred to as Jason-CS) consists of two satellites, the first launched in 2020 and the second planned for launch 2026. Although primarily an altimetry mission, with orbital characteristics fit to that purpose, the Sentinel-6 satellites will also carry GNSS-RO instruments with heritage from the RO instruments onboard the COSMIC-2 satellites. The Sentinel-6 orbits are not Sun-synchronous, have a medium inclination, and the altitudes are higher than for the other RO missions (about 1300 km instead of the typical 600-800 km). The value of the Sentinel-6 RO data from a climate science perspective is primarily in providing data covering the full diurnal cycle, and in boosting the overall data numbers.
10.2. 3.6.2 Water Vapour TCWV WV_cci/CM SAF (COMBI) TCDR v1.0
The heritage of the MERIS instrument is currently flying in the form of OLCI on the Sentinel 3 satellites. The opportunities do not end there: the microwave radiometer (MWR) onboard Sentinel 3 is primarily used to support the Synthetic Aperture Radar Altimeter (SRAL). The MWR provides cloud liquid water and water vapor and is thus an additional source of TCWV data over the ocean and is available for, e.g., cross validation.
Furthermore, the current Sentinel 5-P and the future Sentinel 4 and Sentinel 5 missions as well as the high-resolution IR sounder (IASI) and IASI-NG (next generation) all have band configurations that provide information about the vertical profile and column, too. Most of them could be used in a similar way to how the TCWV WV_cci/CM SAF (COMBI) algorithm is used for MERIS/MODIS/OLCI. Primarily the H2O absorption peaks would lie in different spectral ranges.
A real “game changer” for TCWV retrieval in the NIR will be MTG. MTG will carry the Flexible Combined Imager (FCI) which features band configurations in the NIR in an absorption peak of gaseous H2O. This would make it a formidable platform to observe water vapour at hourly or even sub-hourly frequencies at a reasonable resolution.
References
Diedrich, H., Preusker, R., Lindstrot, R., and Fischer, J.: Retrieval of daytime total columnar water vapour from MODIS measurements over land surfaces, Atm. Meas. Tech., 8, 823-836, doi:10.5194/amt-8-823-2015 , 2015.
This document has been produced in the context of the Copernicus Climate Change Service (C3S).
The activities leading to these results have been contracted by the European Centre for Medium-Range Weather Forecasts, operator of C3S on behalf of the European Union (Delegation Agreement signed on 11/11/2014 and Contribution Agreement signed on 22/07/2021). All information in this document is provided "as is" and no guarantee or warranty is given that the information is fit for any particular purpose.
The users thereof use the information at their sole risk and liability. For the avoidance of all doubt , the European Commission and the European Centre for Medium - Range Weather Forecasts have no liability in respect of this document, which is merely representing the author's view.
