Contributors: W. Dorigo (WD) (tU Wien), T. Scanlon (TS) (tU Wien), W. Preimesberger (WP) (tU Wien), P. Buttinger (PB) (tU Wien), A. Pasik (AP) (tU Wien), R. Kidd (RK) (EODC), C. Chatzikyriakou (CC) (EODC)
Issued by: EODC/Richard Kidd
Date: 29/05/2020
Ref: C3S_312b_Lot4.D2.SM.1-v2.0_202001_Product_Quality_Assurance_Document_v1.0
Official reference number service contract: 2018/C3S_312b_Lot4_EODC/SC2
History of modifications
List of datasets covered by this document
Related documents
Acronyms
General definitions
Accuracy: The closeness of agreement between a measured quantity value and a true quantity value of a measurand (JCGM, 2008). The metrics used here to represent accuracy are correlation and unbiased Root Mean Square Difference (ubRMSD). These metrics are commonly used throughout the scientific community as measures of accuracy (Entekhabi et al., 2010).
Bias: Estimate of a systematic measurement error (JCGM, 2008).
Error: Measured quantity value minus a reference quantity value (JCGM, 2008).
Precision: Closeness of agreement between indications or measured quantity values obtained by replicate measurements on the same or similar objects under specified conditions (JCGM, 2008).
Quality Assurance: Part of quality management focused on providing confidence that quality requirements will be fulfilled (Institution, 2015).
Stability: Property of a measuring instrument whereby its metrological properties remain constant in time (JCGM, 2008). Note that for earth observation activities, "measuring instrument" can be translated as "retrieved variable" and "metrological properties" can be interpreted as "variability of retrieved variable". Alternatively, GCOS (WMO, 2016) defines stability as the extent to which the uncertainty of measurement remains constant in time; the GCOS requirements are stated as the maximum acceptable change in systematic error (usually per decade).
Scope of the document
The purpose of this document is to describe the product Quality Assurance (QA) for the soil moisture products developed by TU Wien, EODC and VanderSat for the Copernicus Climate Change (C3S) service. The production of the products has been funded by the European Centre for Medium Range Weather Forecasting (ECMWF), while the scientific development and processor prototyping have been funded by the Climate Change Initiative (CCI) of the European Space Agency (ESA).
The C3S soil moisture product suite provides PASSIVE, ACTIVE and COMBINED (passive + active) microwave soil moisture products on a daily, dekadal (10-days) and monthly basis. The data is provided in a regular 0.25 degree grid based on the WGS84 reference system. The product is available globally between November 1978 and present day (for both the PASSIVE and COMBINED products) and August 1991 and present day (for the ACTIVE product). For details about the products, we refer to the Product User Guide and Specifications (PUGS) [D1].
This document defines and describes the datasets, validation methods and strategies used for the validation and characterisation of the accuracy and stability of the soil moisture product. Note that, whilst some of the methods described in this document will be implemented routinely each time the product is reprocessed, others will be implemented on an “ad hoc” basis as deemed necessary. The target audience of this document is the users of the C3S soil moisture data products who wish to understand how the results reported in the Product Quality Assessment Report (PQAR) have been derived.
This current document is applicable to the QA activities performed on the version of the Climate Data Record (CDR) produced in January 2020 (filenames have the extension XXX-v201912.0.0.nc). Currently, the methodology does not include details of how the Intermediate Climate Data Records (ICDRs) will be assessed, since no suitable quality controlled near-real-time (NRT) reference data are available at this time. This may change with the envisaged NRT availability of ERA5 soil moisture fields. However, it is noted that, to achieve maximum consistency between CDR and ICDR, both products use the same Level 2 products (based on NRT data streams) and thus have very similar quality characteristics. The methods described here are implemented on the ACTIVE, PASSIVE and COMBINED daily, dekadal (10-daily) and monthly products.
Executive summary
The purpose of this document (the Product Quality Assurance Document (PQAD)) is to describe the product Quality Assurance (QA) for the soil moisture products developed by TU Wien, EODC and VanderSat for the Copernicus Climate Change (C3S) service. The production of the product has been funded by the European Centre for Medium Range Weather Forecasting (ECMWF), while the scientific development and processor prototyping have been funded by the Climate Change Initiative (CCI) of the European Space Agency (ESA).
The Product Quality Assurance includes the definition and description of the datasets, validation methods and strategies used for the validation and characterisation of the accuracy and stability of the soil moisture product. This document is applicable to the QA activities performed on the version of the Climate Data Record (CDR) produced in January 2020 (filenames have the extension XXX-v201912.0.0.nc). Currently, the methodology does not include details of how the Interim Climate Data Records (ICDRs) will be assessed, since until now no suitable quality controlled near-real-time (NRT) reference data have been identified. This may change with the envisaged NRT availability of ERA5 soil moisture in the near future (currently data is available from 2008 to within 3 months of real time). Note that, to achieve maximum consistency between CDR and ICDR, both products use the same Level 2 products (based on NRT data streams) and thus have very similar quality characteristics.
The QA methodology broadly comprises the following parts: accuracy assessment, stability assessment, a completeness / consistency assessment (spatial and temporal), visual inspection of the product, demonstration of uncertainty analysis and comparison to previous versions of the product. The first two sections focus on demonstrating that the Key Performance Indicators (KPIs) for the product are met. Note these KPIs take into account Global Climate Observing System (GCOS) and user requirements for the product.
The accuracy assessment is based on the comparison of the C3S soil moisture products to ground reference data from the International Soil Moisture Network (ISMN) as well as the comparison to Land Surface Models (LSMs) including the Global Land Data Assimilation System (GLDAS) and ERA datasets.
The stability assessment uses both accuracy metric monitoring and the output of break-point detection methods as a proxy for stability measures. Further research on robust methods for the derivation of stability metrics is currently underway.
The completeness / consistency assessment considers both the temporal and spatial domain, focussing on the coverage of the dataset and taking into consideration whether or not the observations were flagged as valid or not.
The visual inspection of the dataset focusses on presenting timeseries and daily global maps of the dataset. Whilst such checks are simple, they do provide insight into the attributes of the dataset and allow simple verification of the data product.
The demonstration of uncertainty estimates (which are based on a combination of triple collocation analysis and error propagation) focuses on showing the evolution of uncertainties over the time series by providing Hovmöeller diagrams of the uncertainties.
The current version of the C3S product (v201912) is compared previous versions (including any intermediate products). The assessment focusses on the differences between the products and how these can be attributed to changes in the input datasets as well as changes in the algorithm. The product is also compared to the ESA CCI SM v04.5 product which is based upon the same algorithm as the C3S CDR v2.0 product (v201912). In this case underlying algorithm common between CDR v2.0 and the ESA CCI SM v04.5 product is the ESA CCI SM v04.4 algorithm.
This document represents the methodology for the current C3S soil moisture product (v201912). The document will be updated as necessary for future releases of the data product.
1. Validated products
1.1. The C3S Soil Moisture Product
C3S soil moisture provides passive (named PASSIVE), active (ACTIVE) and passive + active (COMBINED) soil moisture products on a daily, dekadal (10-days) and monthly basis. The data is provided in a regular 0.25 degree grid based on the WGS84 reference system. The product is available globally between November 1978 and present day (for PASSIVE and COMBINED) and August 1991 and present day (for ACTIVE). The product has been produced by TU Wien, Vandersat and EODC.
The C3S soil moisture product is generated from a set of passive microwave radiometers (SMMR, SSM/I, TMI, AMSR-E, WindSat, AMSR2 and SMOS) and active microwave scatterometers (ERS-1/2 AMI WS and ASCAT (Metop-A and Metop-B)). The “ACTIVE product” and the “PASSIVE product” are created by fusing scatterometer and radiometer soil moisture products, respectively. The “COMBINED product” is a blended product based on the two former datasets. Data files are provided as NetCDF-4 classic format and comply with CF-1.6 conventions1. A summary of the specification for the C3S soil moisture products are provided in Table 1.
A detailed description of the product generation is provided in the Algorithm Theoretical Basis Document (ATBD) [D2] with further information on the product given in the Product User Guide and Specifications (PUGS) [D1]. The underlying algorithm is based on that used in the generation of the ESA CCI v04.4product with the which is described in relevant documents (Dorigo et al., 2017, Gruber et al., 2019., Gruber et al., 2017, Chung et al., 2018, Liu et al., 2012). In addition, detailed provenance traceability information can be found in the metadata of the product (i.e. in the global attributes of the netCDF daily image files).
Table 1: Product Specifications for the C3S Soil Moisture Product
Requirement | Target for C3S Soil Moisture Product |
|---|---|
Parameter of interest | Volumetric surface soil moisture |
Unit | Volumetric (m3 / m3) (PASSIVE and COMBINED products) |
Product aggregation | Gridded Level 2 (L2) single sensor products |
Native spatial resolution | 25-50 km |
Product spatial sampling | 0.25° |
Record length CDR | > 40 years (1978/11 – running present; PASSIVE and COMBINED); > 28 years (1991/06 – running present; ACTIVE) |
Record length ICDR | > 10 days (2020/01 – running present) |
Native temporal resolution | 0.5-2 days |
Product temporal sampling | 1 day |
Product accuracy | Variable (0.01 – 0.10 m3 / m3) depending on land cover and climate |
Product stability | 0.01 m3 / m3 / y |
Quality Flags | Frozen soil, snow coverage, dense vegetation, retrieval failure, sensor used for each observation, overpass mode, overpass time, Radio Frequency Interference (RFI) |
Uncertainty | Daily estimate, per pixel |
1.2. Available Products
The C3S soil moisture product comprises a long-term data record called a CDR which runs from 1978 (passive and combined) or 1991 (active) to the end of December 2019. This CDR is updated every dekade (approximately every 10 days) in an appended dataset called an ICDR. The theoretical algorithm and the processing implemented in the CDRs and ICDRs are exactly the same and the data provided is consistent between them. A new version of the CDR may be produced under the following cases:
- There are updates to the algorithm (scientific advances, for example)
- Processing parameters are updated.
- New sensors are added to the algorithm.
- Any Near Real Time (NRT) products are changed making a reprocessing of the archive necessary for consistency.
This current document is applicable to the QA activities performed on the version of the CDR produced in January 2020 (v201912). This includes all products ACTIVE, PASSIVE, and COMBINED at daily, dekadal and monthly temporal resolutions.
1.3. Soil Moisture Parameters and Units
The C3S soil moisture products along with the associated uncertainties (for the daily product only) and various other data are provided for each 0.25 degree grid cell globally on a daily, dekadal or monthly basis (depending on the product). Full details of the product may be found in the PUGS [D1].
The exact depth that the soil moisture product represents is dependent on many factors including the properties of the soil (physical and dielectric) as well as the characteristics of the sensors used to measure the soil moisture. Therefore, an exact depth is not currently attributed to either the product as a whole or to each individual pixel.
For the passive and combined soil moisture products, the volumetric soil moisture is provided in units of [m3 / m3] (volumetric soil moisture). For the active product, the soil moisture is expressed as degree of saturation [%]. Porosity expressed as volumetric soil moisture units may be used to enable the conversion to the volumetric soil moisture (see Hillel 2004).
2. Description of reference datasets
2.1. International Soil Moisture Network (ISMN)
The ISMN (Dorigo et al., 2011, Dorigo et al., 2013) has been established as a centralised data-hosting facility where globally available in-situ soil moisture measurements from operational networks and validation campaigns are collected, harmonised, and made available to users2 . It exists as a means for the geo-scientific community to validate and improve global satellite observations and modelled products. The network is coordinated by the Global Energy and Water Cycle Experiment (GEWEX) in cooperation with Group on Earth Observations (GEO) and CEOS (Committee for Earth Observation Satellites). The measurements contributing to the ISMN are heterogeneous in that the technique, depth represented, and other factors, may vary within the network. The locations of the ISMN sites are shown in Figure 1. As of February 2020 ISMN, database integrates data from 61 networks (2609 stations).
Figure 1: Site locations of datasets distributed by the ISMN (as of February 2020). Taken from the ISMN data viewer: http://www.geo.tuwien.ac.at/insitu/data_viewer/ISMN.php.
The data available within the ISMN is subject to quality controls (detailed in (Dorigo et al., 2013)) and provided with quality flags. The quality controls include an assessment against a possible range of important metrological variables which are applied equally to all datasets.
The ISMN dataset has been utilised in the validation of the ESA CCI product, with the most extensive evaluation being undertaken by (Dorigo et al., 2015), who employed all useable observations from the ISMN (Dorigo et al., 2017).
Figure 2: Temporal availability of in-situ measurements from networks within the ISMN database, compared to modelled outputs and EO sensor derivatives (as of February 2020). Active networks are those which continue to contribute to the ISMN; inactive are networks for which no further updates are expected.
2.2. ERA5
ERA53 (Hersbach et al, 2018) provides global estimates of variables including soil moisture by combining historical observations with advanced modelling and data assimilation systems. Currently, the data record is available from 1979 up to within 5 days of real time. The long term dataset from 1950 onwards is expected to be available in 2020.
2.3. ERA5-Land
ERA5-Land is the successor to the previously used ERA-Interim/Land reanalysis (Balsamo et al, 2012) data product. ERA5-Land has been produced by replaying the land component of the ECMWF ERA5 climate reanalysis. ERA5-Land Volumetric Soil Moisture and Soil Temperature are available with a spatial resolution of 0.1 * 0.1 degrees and representative of Soil Water and Soil Temperature in 4 layers (0-7cm, 7-28cm, 28-100cm, 100-289cm). The top layer is used for comparison with ESA CCI SM. ERA5-Land from January 2001 onward.
2.4. ESA CCI Soil Moisture
The CCI project was initiated in 2009 by ESA in response to the United Nations Framework Convention on Climate Change (UNFCCC) and Global Climate Observing System (GCOS) needs for Essential Climate Variable (ECV) databases (Plummer et al., 2017). In 2012, ESA released the first multi-decadal, global satellite-observed soil moisture dataset, named ESA CCI SM, combining various single-sensor active and passive microwave soil moisture products (Dorigo et al., 2017). The current C3S product (v201912) is based (scientifically, algorithmically and programmatically) on the v04.4 of the ESA CCI SM product (released in November 2018.
3. Description of product quality assurance methodology
3.1. Introduction
QA in the context of Earth Observation (EO) applications, and in particular CDR generation can be defined as the processes undertaken to ensure that the data product meets any defined requirements. QA for a CDR product such as the C3S soil moisture product will generally include:
- Accuracy assessment of the data product, i.e. validation defined by the Land Product Validation (LPV) group4 as: "the process of assessing, by independent means, the quality of the data products derived from the system outputs" (also see Justice et al., 2000).
- Stability assessment of the product over long time periods. This refers to the properties of the product remaining constant in time and has been defined for Earth Observation applications (WMO, 2016) as the extent to which the systematic error associated with the product changes.
- Completeness and consistency checking to demonstrate the continuous nature of the product over the spatial and temporal domains. This includes evaluation of the number of valid observations available in the dataset.
- Visual inspection of the dataset which includes plotting maps and timeseries of the data to allow a check on the spatial and temporal characteristics of the dataset to ensure they are as expected.
- Uncertainty assessment provides plots of the uncertainties associated with the product.
- Comparison to previous products includes an assessment of the dataset against previously released C3S (and CCI) versions to show the evolution of the algorithm over time. The dataset is also compared to the CCI dataset upon which it is based to ensure the dataset has been made as expected.
- Verification of the product to ensure that the outputs generated are as expected (note, for the C3S soil moisture product, such activities are described within the System Quality Assurance Document (SQAD) (Paulik and Reimer, 2018)).
QA of soil moisture datasets is important as quality of individual soil moisture observations can be impacted by numerous factors (Dorigo et al., 2017). These factors can be roughly divided into the following categories: sensor properties, orbital characteristics, environmental conditions, algorithm skill (e.g. methods used to correct for vegetation impacts) and post-processing (e.g. resampling). Further details of each of these characteristics are provided in Table 2 (taken from Dorigo et al., 2017). The majority of these factors add some degree of random error and bias to the obtained estimate (Dorigo et al., 2017).
The QA activities to be undertaken on the C3S soil moisture product include all of the steps listed in 1 – 6 above (for further details see Sections 3.6 to 3.9). However, the focus is on determining the accuracy (see Section 3.4) and the stability (see Section 3.5) of the product with respect to defined requirements (see Section 3.2).
The current QA activities will focus on the assessment at the product timescales (daily, dekadal and monthly) as well as three-monthly, four yearly and using different aggregation periods dependent on the sensor merging periods used in the product. Additional work considering inter-annual and seasonal variability will be considered in the QA of later versions of this product. The same measures, as well as pre-processing steps to be applied in the assessments, will be implemented consistently between scales; these are described in Section 3.3.
Ideally, the assessments would be performed on different spatial and temporal scales, with the same evaluation measures applied at each scale. Currently, the focus is on the global dataset, with the effect of different land cover and climate types considered where data is accessible.
A similar assessment methodology to that presented here has been previously utilised in the ESA CCI soil moisture project (Mittelbach et al., 2014, Mittelbach et al., 2012). This validation methodology was subject to user community acceptance prior to use and as such, it allowed a contribution to the definition of international standards in the soil moisture domain.
The methods for quality assessment of biogeophysical variables have been developed over several decades and there is significant research available on good practices and techniques (Loew et al., 2017, Gruber et al., 2016). Good practice specifically focussing on soil moisture validation is currently being coordinated at TU Wien, with support of large part of the scientific community. The available guidance is taken into account within the methodology. This is complemented by the development of the Quality Assurance for Soil Moisture (QA4SM)5 platform which provides robust, traceable validation of different data products against reference data including ISMN, GLDAS and ERA5/ERA5-Land.
The evaluation of the quality of the dataset should be continuously repeated once a new dataset version becomes available to assess the potential impact of improved calibrations and algorithmic changes (Dorigo et al., 2017). The methodology presented here is applied to each dataset version, however there may be updates in the future as quality assessment methods are improved / advanced.
Table 2: Main sensor, observational and environmental factors impacting the quality of the C3S soil moisture products. Taken from (Dorigo et al., 2017).
Factor | Category | Affects active (A) or passive (P) observations | Impact on soil moisture retrieval | How it is handled in the C3S product and potential recommendation(s) for use |
|---|---|---|---|---|
Observation frequency / wavelength | Sensor | A,P | Shorter wavelengths (higher frequencies) are more sensitive to vegetation, theoretically causing higher errors. Different wavelengths have different soil penetration depths, and thus represent different surface soil moisture columns. | Preferential use of longer wavelengths when multiple frequencies are available. Indirectly accounted for by Signal to Noise Ratio (SNR)-based weighting and indirectly quantified as part of the random error estimate (see below). The frequency and sensor that were used in the product generation are provided as ancillary data. |
Instrument errors and noise | Sensor | A,P | Directly impacts the error of the single-sensor soil moisture retrieval. | Included in total random error assessed by triple collocation. Soil moisture random error provided as a separate variable in product. |
Local incidence angle and azimuth | Sensor | A | Impacts backscatter signal strength and hence retrieved value. | Accounted for by incidence angle and azimuthal correction in Level 2 retrieval. Remaining uncertainty is indirectly quantified as part of random error estimate. |
Local observation time | Orbital | A,P | Vegetation water content changes during the day (Steele-Dunne et al., 2012), but this variability is not accounted for by the retrieval models. Early morning observations may be influenced by dew on soil and vegetation, thus leading to higher observed soil moisture. Solar irradiation causes discrepancies between canopy and soil temperatures which complicate the retrieval of soil moisture (Parinussa et al., 2016); see also "Land surface temperature" below. Intra-daily variations because of convective precipitation and successive evaporation may be missed. | Partly addressed by excluding "day-time" radiometer observations. Remaining uncertainty is indirectly quantified as part of random error estimate. |
Vegetation cover | Environmental | A,P | Reduces signal strength from soil and hence increases uncertainty of soil moisture retrieval. | Included in total random error of product assessed by triple collocation. Dense vegetation is masked for passive Level 2 products according to sensor-specific Vegetation Optical Depth (VOD) thresholds: soil moisture random error is provided as a separate variable. |
Topography | Environmental | A,P | Impacts backscatter signal strength; causes heterogeneous soil moisture conditions within the footprint. | Not accounted for. Topography index is provided as metadata. A flagging of pixels with topography index > 10 % by the data user is recommended. |
Open water | Environmental | A,P | Impacts backscatter and brightness temperature signal strength. | Not accounted for. Open water fraction is provided as metadata. A flagging of pixels with open water fraction > 10 % is by the data user is recommended. |
Urban areas, infrastructure | Environmental | A,P | Impacts backscatter and brightness temperature signal strength. | Not directly account for. Uncertainty is indirectly quantified as part of random error estimate. |
Frozen soil water | Environmental | A,P | Strongly impacts observed backscatter / brightness temperatures causing a "false" reduction in soil moisture. | Masked using radiometer-based land surface temperature observations (Holmes et al., 2009) and freeze / thaw detection (Naeimi et al., 2012) from Level 2 algorithms, and ancillary data from ERA5 and GLDAS-Noah in product generation. Flag provided as metadata. |
Dry soil scattering | Environmental | A | Volume scattering causes unrealistic rises in retrieved soil moisture (Wagner et al., 2013). | Nor directly accounted for, but indirectly account for by low weight (related to high error) received in SNR-based blending. |
Land surface temperature | Environmental | P | Errors in land surface temperature directly impact the quality of surface soil moisture retrievals. | Partly addressed by excluding "day-time" radiometer observations. Remaining uncertainty is indirectly quantified as part of random error estimate. |
Radio frequency interference (passive only) | Environmental | P | Artificially emitted radiance increases brightness temperatures and, hence, leads to a dry bias in retrieved soil moisture. | In the case of multi-frequency radiometers, a higher frequency channel (e.g. X-band) is used if RFI is detected. In other cases, the observation is masked. |
3.2. Product Quality Requirements
The QA process is required to ensure that the soil moisture product meets any requirements which have been set out prior to the development of the product. This section details the Key Performance Indicators (KPIs) for the C3S product (Table 3). These KPIs have been developed taking into account the user requirements from the CCI soil moisture product (Mittelbach et al., 2012) as well as the GCOS requirements (WMO, 2016). Note that, for accuracy, the KPIs match the GCOS requirements, but for stability, the GCOS requirements are slightly more stringent (0.04 m³ / m³ / y). The metrics used here to represent accuracy are Spearman rank correlation and unbiased Root Mean Square Difference (ubRMSD). These metrics are commonly used throughout the scientific community as measures of accuracy (Entekhabi et al., 2010, Gruber et al., 2017).
Note that "in the latest quarter" in Table 3 means the last three months of the product which is available. The assessment method presented here focusses on the CDRs which are generated once (or perhaps twice) a year. The assessment method may be updated to account for this three-monthly assessment within the lifetime of the service, however, this requires the availability of quality-controlled NRT soil moisture data which can be used as a reference.
Table 3: Key Performance Indicators for the C3S Soil Moisture Product
KPI # | KPI Title | Performance Target and Unit of Measure |
|---|---|---|
Accuracy KPIs | ||
KPI.D1.1 | CDR Radiometer with a daily resolution in latest quarter | Variable (0.01-0.10 m³ / m³), depending on land cover and climate |
KPI.D2.1 | CDR Scatterometer with a daily resolution in latest quarter | |
KPI.D3.1 | CDR Combined with a daily resolution in latest quarter | |
KPI.D4.1 | CDR Radiometer with a daily resolution in latest quarter | |
KPI.D5.1 | CDR Scatterometer with a daily resolution in latest quarter | |
KPI.D6.1 | CDR Combined with a daily resolution in latest quarter | |
Stability KPIs | ||
KPI.D1.2 | CDR Radiometer with a daily resolution in latest quarter | 0.05 m³ / m³ / y (1) |
KPI.D2.2 | CDR Scatterometer with a daily resolution in latest quarter | |
KPI.D3.2 | CDR Combined with a daily resolution in latest quarter | |
KPI.D4.2 | ICDR Radiometer with a daily resolution in latest quarter | |
KPI.D5.2 | ICDR Scatterometer with a daily resolution in latest quarter | |
KPI.D6.2 | ICDR Combined with a daily resolution in latest quarter | |
(1)Work on the metrics used for stability assessment is ongoing with the aim of demonstrating compliance with these performance targets.
3.3. General Evaluation Methods
3.3.1. Pre-Processing
This section discusses how the different datasets are pre-processed to ensure that the parameters being compared are as equivalent as possible, for example, in-situ compared with satellite datasets can have large representativeness errors (Gruber et al., 2013) which may impact any comparisons undertaken (Su et al., 2016) and these need to be accounted for as far as possible.
Masking
Masking of any of the datasets used will be undertaken following the guidelines set out for the use of their own quality flags. For example, for the ISMN data, flags are available which indicate if the data is "good" (see Dorigo et al., 2013 for more details). The masking applied to the C3S data product will be dependent on the individual assessments being undertaken. Further details are provided in the individual assessments below.
Spatial Resolution
The spatial resolution of the C3S product is 0.25 degrees, however the reference datasets have a range of spatial resolutions from point scale upwards. Therefore, significant consideration should be given to bridging the differences in spatial scale between the datasets. This is particularly important as the spatial variability of the soil moisture can be significant due to complex interactions between pedologic, topographic, vegetative and meteorological factors (Crow et al., 2012).
Currently, the nearest-neighbour approach is used, i.e. the latitude and longitude of the reference dataset pixel or point measurement is used to find the nearest grid point in the C3S dataset. The spatial representativeness of the point data (for example ISMN data) will need to be considered in future iterations of this assessment methodology.
Currently, where there is more than one observation within a grid cell, the observations are averaged to provide a representative soil moisture value, however, the validity of this approach will need to be further assessed. Where only one value is available, this is taken as the value for that grid cell.
In future iterations, a more complex strategy shall be used to derive a statistically representative comparative value. To achieve the optimal practical solution, upscaling methods (such as upscaling enhancement using time stability concepts, block kriging or land surface modelling) may be implemented in all pixels where representative values need to be derived.
Temporal Resolution
The C3S soil moisture dataset is provided at daily, dekadal (10-daily) and monthly temporal resolutions. The assessments will be undertaken at all time steps. As the comparison should consider the closest temporal measurements, it is expected that the assessment at a daily resolution would provide the best results. Monthly data will be particularly useful where long time series are being processed (for example in the stability assessment) where the aggregation to monthly data has a minimal impact on the results of the assessment.
Soil Moisture Depth
As described in Section 1.3, the exact depth that the soil moisture product represents is not available. Therefore, when considering the use of other products, the upper soil moisture (a depth of 0.5 to 5.0 cm) is usually taken as an appropriate comparable parameter. In the accuracy assessment against ISMN data, two depth ranges of ISMN sensors will be used: 0 to 5 cm and 5 to 10 cm. For the accuracy assessment against GLDAS data, the field 'SoilTMP0_10cm_inst' will be used which provides soil moisture from 0 to 10 cm depth. For ECMWF reanalysis products, the first layer (0-7cm) will be used.
Dynamic Scaling
The different datasets utilised in the quality assessment are available in different dynamic ranges, therefore, scaling is applied to bring the datasets into a common climatology. A mean standard deviation scaling is applied6 .
3.3.2. Evaluation Measures
The methods applied, in particular for the accuracy assessment, focus on the performance of the scaled absolute values of soil moisture (ABS) (scaled as described in Section 3.3.1 above) as well as the resulting temporal short- (STA) and long-term anomalies (Inter-Annual Anomalies (IAA)).
STA are calculated by subtracting a 35-day moving average and dividing by the standard deviation (Albergel et al., 2008). For IAA, the climatological mean of a specific day is either based on all values of that day of the year, or taking into account a 10-day window around that day to account for potential shortages of data in the specific time period (Nicolai-shaw et al., 2015).
Appropriate statistical measures will be used in the assessment including the Spearman rank and Pearson's correlation and unbiased root mean square difference (ubRMSD). Such measures have been frequently, and successfully, used in previous inter-comparison studies (Rüdiger et al., 2009; Brocca et al., 2011; Gruhier et al., 2009).
3.3.3. Presentation of Results
In general, results of correlations will be presented as box-plots showing the median of the correlation values as well as the 95 % confidence interval. An example is shown in Figure 3. Global maps will also be provided to show how the metrics vary spatially.
Figure 3: Example of boxplots (displaying median, interquartile range (IQR), upper (lower) quartile plus (minus) 1.5 times the IQR, and outliers) of the correlations of the publicly released versions of the ESA CCI SM COMBINED and ERA-Interim / Land with globally available in-situ probe observations down to a maximum depth of 5 cm, both for absolute values and long-term soil moisture anomalies. Only observations within the period 1991-2010 were considered. Taken from (Dorigo et al., 2017).
3.4. Accuracy
3.4.1. Introduction
Accuracy assessment is undertaken through the comparison of C3S products against reference datasets. However, as discussed in (Dorigo et al., 2017), the reliability of such comparisons hinges on the availability of stable, long-term reference datasets, something which is currently still lacking (WMO, 2016).
Here, the datasets used for the accuracy assessment are the ISMN, and ERA5/ERA5-Land datasets (described further in Section 2). These have been chosen due to their availability over a relatively long time period (albeit with gaps in some periods at some locations for the in-situ data) and because they are publicly available or available on request, enabling traceability of the datasets as well as allowing the validation results to be reproduced by a third party.
3.4.2. Point Scale
The point scale accuracy assessment is undertaken against the ISMN dataset. This type of accuracy assessment is particularly useful as it allows the comparison of instruments which have not been subjected to the rigors of space (radiation, launch forces etc.) with data derived from space-borne sensors. The advantage of this approach is that any calibration and characterisation of the in-situ sensors undertaken in the laboratory will likely be representative of the sensor's performance throughout its life-cycle. In addition, such sensors can be retrieved from the field and routinely re-calibrated / re-characterised as necessary, resulting in ongoing traceability of the sensors.
Table 4: Settings used in the assessment of the C3S soil moisture product against the ISMN
Setting | Details |
|---|---|
Temporal Matching | A temporal window of 12 hours is used to find matches between the C3S and ISMN datasets, i.e. the ISMN may be from 6 pm to 6 am around midnight UTC (timestamp for each C3S daily image). |
Spatial Matching | The nearest land Grid Point Index (gpi) from the grid C3S data is found using the lon/lat of the ISMN station metadata. Where a single C3S gpi is associated with more than one ISMN station, an unweighted average of the ISMN data is taken (i.e. average for each timestamp) prior to calculation of the metrics). |
Scaling | The C3S data is scaled to the ISMN data using mean – standard deviation scaling. |
Filters | The ISMN data has been filtered on the "soil moisture_flag" column such that only observations marked "G" are utilised. The depths of the ISMN used vary for each assessment; this is stated in the information below. |
The settings used in the ISMN assessment are summarised in Table 4. The assessment will be undertaken using all ISMN stations with available data flagged as "good" within the time period covered by the C3S product Full details of ISMN quality flags is available here: https://ismn.geo.tuwien.ac.at/data-access/quality-flags/. A set of ISMN networks have been chosen based on previous experience with, and knowledge of the networks; a full list of the stations used is provided in Table 5. From these networks, the ISMN stations used are selected and compiled taking into account the pre-processing steps outlined in Section 3.3.1. There is no need to mask the C3S data used as the final daily, dekadal and monthly images only contain data which is flagged as 'Good' within the product and the timeseries files used for the assessment are generated directly from these image files.
Table 5: ISMN stations used in the accuracy assessment of the C3S product
Network | Country | No. of Stations |
|---|---|---|
AMMA-CATCH | Benin, Niger, Mali | 7 |
BIEBRZA_S-1 | Poland | 30 |
BNZ-LTER | Alaska | 12 |
CARBOAFRICA | Sudan | 1 |
COSMOS | USA | 109 |
CTP_SMTMN | China | 57 |
DAHRA | Senegal | 1 |
FLUXNET-AMERIFLUX | USA | 2 |
FMI | Finland | 27 |
FR_Aqui | France | 5 |
HOBE | Denmark | 32 |
iRON | USA | 9 |
LAB-net | Chile | 3 |
MySMNet | Malaysia | 7 |
OZNET | Australia | 38 |
PBO_H2O | USA | 159 |
REMEDHUS | Spain | 24 |
RISMA | Canada | 23 |
RSMN | Romania | 20 |
SCAN | USA | 239 |
SMOSMANIA | France | 22 |
TERENO | Germany | 5 |
USCRN | USA | 115 |
WEGENERNET | Austria | 12 |
WSMN | UK | 8 |
Once both datasets have been masked using their own flags, the data is matched based on timestamps using a nearest-neighbour search with a 12 hour matching window. The ISMN data is then scaled the C3S data using the scaling methods described above. The scaled timeseries of the in-situ observations are then compared with the associated C3S grid cell and correlation and ubRMSD are calculated. As well as the global statistics being presented, the results are also aggregated for different attributes of the soil moisture data: depth, texture, Köppen-Geiger classes and land cover. The data allowing these stratifications of the results are provided within the ISMN dataset (Dorigo et al., 2011). In the future, the stratification of the results will be considered further. This may include splitting the results by factors which may affect the C3S soil moisture product including vegetation-cover and the proportional composition of land surface features within each pixel.
A specific study may be undertaken in the future considering boreal and sub-arctic environments. The boreal environment poses distinctive challenges for soil moisture retrievals. For example, long periods of frozen soil and snow accumulation as well as thick layers of organic soil (boreal forests in southern latitudes in particular contain relatively large fractions of peatland) can affect the retrievals. Furthermore, the northern latitudes have typically been very sparsely covered with in-situ soil moisture observations. Therefore, validation using point scale observations may not be sufficient for sub-arctic environments. For improved validation in these areas, land surface models will be introduced in order to scale the in-situ reference measurements from point observations to larger regional areas.
3.4.3. Regional and Global Scale
Regional and global datasets will be used to quantify the performance of the retrieval algorithms on a larger scale than the point scale measurements. While detailed information is provided by networks such as the ISMN, ground based observations lack sufficient global coverage and consistency for comprehensive earth system assessments (Dorigo et al., 2017). Therefore, reanalysis LSM products are used to allow the comparison of the relative values of the C3S product over a larger domain, i.e. global scales and for specific regions as well as over a long time period (Albergel et al., 2013). ERA5/ERA5-Land will be used for this purpose at v201912 of the dataset (to allow comparison of the results from the two). In addition, the GLDAS dataset (version 2.1) will be used however, the dataset is expected to match the COMBINED product well as the current algorithm uses scaling to GLDAS to bring the datasets to a common scale.
3.5. Stability
The stability of ECV products is a topic of research and providing a metric which describes stability in terms of change in the uncertainty of the variable of interest per decade is currently being investigated. Therefore, we present the results of two methods: one based on monitoring the evolution of accuracy metrics over time and the other focusing on the identification of structural breaks in the timeseries.
3.5.1. Monitoring Accuracy Metrics
To monitor stability, accuracy metrics are calculated for the C3S data compared against ISMN data for individual time periods of either 1 or 3 years. The trends in the metrics are then used to assess the stability of the product. For example, we can calculate the theil-sen estimate of the slope using the means of the correlation and ubRMSD over time.
This method assumes that the uncertainty associated with the data is characterised completely by the comparison to reference data, which is unlikely to be the case. A more thorough approach might include extracting the systematic component of the uncertainty over time and assessing the trends in this variable. Such an approach is currently under investigation and will be included in the PQAR [D3] if methodology is sufficiently developed.
3.5.2. Break Detection Methods
The break detection will focus on the identification of inhomogeneities in the dataset, aiming to characterise their extent (similar to tests performed in Su et al. 2016 and Preimesberger et al. 2017). Inhomogeneities are defined as artificial discontinuities in the absolute soil moisture level or in the variability of the soil moisture (Su et al., 2016). These are most likely to arise due to changing observation sensors or changes in the retrieval algorithms.
To test for inhomogeneities, the MERRA-2 data will be compared to the C3S soil moisture product over a long time period, which in this case is the entire times-series of the C3S soil moisture product, see Figure 4 below. A difference time series will be produced for the time-frame and the Wilcoxon (WK) rank sum test and the Fligner-Killeen (FK) variance test will be used to identify break points. The WK rank sum test compares the statistical significance of correspondence in distributions of two ordered and ranked datasets. The FK variance test will test of significant differences in variance before / after the break time.
The inhomogeneity testing is achieved by identifying potential locations of expected break times in the time series (for example where a change in sensors used occurs – see Figure 4). The WK and FK are then derived using the monthly averages of soil moisture for each time period (between potential break times). Where WK or FK detects a break with a significance threshold of p<0.01 it is considered that this indicates a potential discontinuity in the time-series.
The metrics provided by the WK and FK testing are (for each expected break time) a Boolean value indicating whether or not a break has been detected, the number of observations from the difference time series used before and after the expected break time and other information on the statistical reliability of the break-point detection.
The stability will be expressed in terms of the longest “stable” time-period within the dataset for each pixel as well as the difference in the differences of the mean of the candidate dataset before and after adjustment. These two measures give a qualitative indication of the stability of the dataset. Work is ongoing to define a robust quantitative measure of stability in the units of m3 / m3 / y for the entire dataset, thereby allowing demonstration against the KPIs (see Section 3.2).
For the stability testing, trends derived from the regional and global datasets will be evaluated against trends from other soil moisture products on monthly, seasonal and yearly timescales. To ensure the robustness of the trend analysis potential sources of uncertainty in the observed trends will be considered taking into account the sign, magnitude and significance of each trend.
Figure 4: Sensor sampling periods (SSP) in the COMBINED product, which are the basis of the break-point detection
3.6. Spatial and Temporal Completeness
In addition to accuracy, stability, uncertainty analysis and determining if the product is within expected boundaries based on other similar products, there are several other potential indicators of a product's quality.
For example, the spatial and temporal completeness provides important quality information for many users. As part of the quality assessment, such factors are considered and reported upon. In addition, it is demonstrated how the currently product compares to previous versions of the product in terms of these attributes, if applicable.
The results are presented in the form of Hovmöeller diagrams of the valid observations, providing a summary of the fraction of valid observations per latitude. In addition, the fraction of valid observations is also plotted on global maps for different periods within the dataset (based on the merging periods of the different sensors).
3.7. Visual Assessment
Analysis of timeseries from a small number of locations provides an insight into the behaviour of the product for different climate and land cover types. Five points have been chosen for which ISMN in-situ data is available (and were used in the above assessment). Details of the points are provided in Table 6 and are shown on a global map in Figure 5. In addition to timeseries analysis, individual dates of daily images will also be inspected.
Table 6: Details of locations chosen for time series analysis.
Ancillary | C3S data location | ISMN station location | |||||
|---|---|---|---|---|---|---|---|
Climate class | Land cover class | Country | gpi | Lat | Lon | Lat | Lon |
Dsc | Sparse vegetation | USA | 890047 | 64.625 | -148.125 | 64.7232 | -148.151 |
Cfa | Cropland | Australia | 316669 | -35.125 | 147.375 | -35.1249 | 147.4974 |
BSk | Cropland | Spain | 756697 | 41.375 | -5.625 | 41.2747 | -5.5919 |
Cfb | Grassland | Germany | 810025 | 50.625 | 6.375 | 50.5149 | 6.3756 |
Cfa | Broadleaf forest | USA | 733335 | 37.375 | -86.125 | 37.2504 | -86.2325 |
Note: all are classified as having 'medium' soil texture. 
Figure 5: Locations of the points used in the time series comparison (points are given at the C3S gpi location).
3.8. Uncertainty Analysis
The C3S soil moisture product is generated with associated uncertainty estimates. These estimates are based on the propagation of uncertainties from the Level 2 to the Level 3 products. This process is described within the ATBD [D2].
Part of the generation process for the product depends on the use of triple collocation analysis which takes estimates the uncertainty with the Level 2 product. Triple collocation analysis may be used either on the local, regional or global scales. The aim of the analysis is to provide an estimate of the variance of the error term associated with a set of measurements (Gruber et al., 2016). Within the C3S project, triple collocation analysis is used to determine the weightings assigned to each available sensor for a particular date / time for combination.
The triple collocation technique does not require the specification of a “true” reference dataset and instead permits the estimation of the error variable of each sensor provided certain assumptions about the error structure are met (Zwieback et al., 2012); the dependency of the method on these assumptions is considered in recent work (Gruber et al., 2016). The triple collocation technique assumes that there are three independent sets of measurements describing the same measurement, for example, soil moisture variations over a specific location. It is assumed that the measurement is linked to the true soil moisture value by an additive and multiplicative term together with a random error. The random uncertainty component provided by this process can be expressed as the SNR which provides useful information by relating the uncertainty to the underlying signal strength.
Within the assessment of the product, global SNR maps for different merging periods will be presented and discussed. In addition, a Hovmöeller diagram of the relative uncertainty (%) will be presented. The latter will provide information both on the availability of uncertainty information within the product as well as the magnitude of the uncertainties over different time periods and latitude bands. In addition, daily images of relative uncertainty will be considered.
3.9. Comparison to Previous Versions
The comparison of the C3S soil moisture product to previous versions of the same product can be useful for ensuring that there are no unexpected, unrealistic or unphysical changes within the individual data points and over the time series. In essence, these comparisons can act as a "sanity check" for the data and can provide a useful insight into the comparative performance of different soil moisture dataset releases. A particular focus will be given to assessing the differences between the C3S and ESA CCI product generated with the same algorithm (but different input ASCAT data streams).
The comparison between different versions is undertaken for each of the quality aspects discussed within this report, i.e. accuracy assessment is performed on the different versions, the spatial and temporal completeness of the products are compared between products to determine if the data coverage of the product is improving, and so on.
4. Summary of most recent validation results
In this section some examples of the validation methodology and their results are provided in relation to the v201812 (CDR v1.0). The actual validation results of v201912 (CDR v2.0) and their in-depth analysis is available in the PQAR document [D3].
The validation of v201912 CDR was completed in April 2020, however, since there were no algorithmic changes implemented between the two product versions, the results presented below effectively demonstrate the validation of the methodology used in the analysis of v201912. The PQAR document for v201912 is expected to become available in May 2020.
4.1. Accuracy – Comparison against ISMN
The results of the comparison to ISMN are shown for different climate classes and land cover types in Figure 6 and Figure 7 respectively.
The results are expected to be in line with other versions of the data set (both C3S and CCI products). The ubRMSD is shown to vary between different climate and land cover classes. In terms of correlation, Csx/Dsx and cropland perform best. In terms of ubRMSD, other climate classes and urban perform best. The highest performance from urban areas is unexpected due to the issues faced in these areas. This will be further investigated in the full PQAR for this product.
Figure 6: Correlation and ubRMSD between C3S v2018012 and ISMN for different climate classes. The boxplots show the mean value and interquartile range.
Figure 7: Correlation and ubRMSD between C3S v201812 and ISMN for different land cover classes. The boxplots show the mean value and interquartile range.
4.2. Accuracy – Comparison against Land Surface Models
The COMBINED product has been compared to GLDAS v2.1 and the correlation is shown in Figure 8. The correlation shows the expected spatial patterns with higher correlations seen in mid- to low-latitudes and negative correlations in areas where there is snow or ice cover for many days of the year. The patterns seen here are similar to those in other data products (both C3S and CCI).
Figure 8: Correlation (Pearson) of the C3S v201812 COMBINED product with GLDAS v2.1 (covering the time period 2000-01-01 to 2018-12-31.
4.3. Stability – Accuracy Metric Trends
To monitor stability, comparison metrics are calculated against ISMN data for individual time periods (see Figure 9) (in the example shown here, 3 years is used). The trends in the metrics could then be used to assess the stability of the product. For example, we can calculate the theil-sen estimate of the slope using the means of the correlation and ubRMSD over time. For the example presented here, this results in the following:
- Correlation slope over time: 0.049 / year
- ubRMSD slope over time: 0.001 / year
The ubRMSD can be interpreted as a stability of 0.01 / decade, which is within the KPI of 0.04 / decade.
The methods for calculating the stability will be considered further in the PQAR for this product.
Figure 9: Stability monitoring through trend assessment of correlation (Pearson) (left) and ubRMSD (right) using ISMN reference data. For this assessment, the C3S and ISMN data were divided into 3 year subset periods.
4.4. Spatial and temporal completeness and consistency
The number of valid (unflagged) observations available is shown globally for the ASCAT / SMOS / AMSR2 merging period (Figure 10) and per latitude for the entire data product period (Figure 11).
The figures show that the coverage is better in Europe, South Africa and the continental US than some in other parts of the world as well as the improvement in the availability of data post-2007 as new sensors became available (see Figure 4). This is as expected for the product due to the orbital paths of the satellites resulting in higher coverage in equatorial regions. The reduced coverage in boreal and tropical region is as expected due to the high VOD expected in these areas. In addition, the coverage of the northern most latitudes in snow and ice for long periods also reduces the coverage in these areas.
Figure 10: Fractional coverage of the C3S COMBINED soil moisture product for the ASCAT / SMOS / AMSR2 period (2012-07-01 to 2018-12-31). Expressed as the total number of daily observations per time period divided by the number of days spanning that time period.
Figure 11: Fraction of days per month with valid (i.e. unflagged) observations of soil moisture for each latitude and time period for the COMBINED product.
4.5. Visual Assessment
The time series for the individual locations for the ACTIVE, PASSIVE and COMBINED products are given in Figure 12. In general, the timeseries appear to follow expected seasonal cycles at each location, i.e. winters are wetter and summers drier and, in the case of gpi 890047 (which is located in Alaska), there are gaps in the data where the location is covered by snow each winter.
Of note for these timeseries, is the sudden drop in the values associated with the PASSIVE product around 2011 which can be seen in all but the cropland land cover type. Whilst this is not so pronounced for the grassland, it is clear in both the sparse vegetation and broadleaf forest. This drop coincides with the drop out of AMSR-E. Further investigation is required to understand this drop and whether this can be seen at other locations. It is noted that the COMBINED product does not suffer such a marked changed despite including the same sensors; this difference may be due to the scaling of the data to GLDAS v2.1 or due to the inclusion of the active sensors in this time period.
Figure 12: Time series comparison for the COMBINED, ACTIVE and PASSIVE products of C3S v201812 for the gpis and land cover types stated for each plot. Note: here, the ACTIVE product is divided by 100 to allow it to be plotted on the same axis as the other products and all products have been temporally resampled to monthly means to allow clearer plotting.
4.6. Uncertainty analysis
The algorithm used to develop the C3S soil moisture product utilises triple collocation analysis to generate weightings for the combination of different soil moisture observations (Gruber et al., 2017). In combination with error propagation techniques, a per-pixel uncertainty is provided within the C3S soil moisture product in the "sm_uncertainty" field.
The relative uncertainty for the date 2018-06-21 is provided as a percentage ((sm_uncertainty / sm) * 100) in Figure 13. This shows the uncertainty for the product is higher in drier areas and lower in those regions were the VOD is higher. Further analysis of the uncertainty associated with the product will be considered in the PQAR.
Figure 13: Daily image of the relative soil moisture uncertainty for the COMBINED product of C3S v2018012. Image date: 2018-06-21.
4.7. Comparison to ESA CCI SM v04.5
C3S v201912 is based on the processor of ESA CCI SM v04.4, which was temporally extended to create the product ESA CCI SM v04.5 in January 2019 (data provided until 2018-12-31). The only difference between the datasets is the ASCAT data which is ingested into the processor; for C3S the NRT data stream is used and for ESA CCI SM the H-SAF H113 and H-SAF 114 have been used.
Due to the use of this different data stream, differences in all the products (ACTIVE, PASSIVE and COMBINED) in the period after the introduction of the ASCAT data is expected. This may be in terms of the data coverage as well as the absolute values of the dataset. While the effect is direct for both the ACTIVE and COMBINED products, in the PASSIVE product this difference is caused by the use of the ASCAT data in the triple collocation process. Here, we concentrate on the effect this difference has on the COMBINED product.
Figure 14: Comparison of the valid observation Hovmöeller diagrams for the COMBINED products of C3S v201812 and ESA CCI SM v04.5.
The number of valid observations in the COMBINED dataset is affected by the difference in the input data stream throughout the product (see Figure 14) with the most extreme differences seen in the ASCAT merging period (2007 onwards). These differences are as expected.
Figure 15: Time series comparison for the COMBINED products of C3S v2018012 and ESA CCI SM v04.5 for the gpis and land cover types stated for each plot.
Time series for the COMBINED dataset have been plotted relative to one another (both timeseries are divided by the C3S v201812 timeseries) (see Figure 15). This figure shows that there are some differences in the products, particularly after 2007 (which is where the input data streams diverge for ASCAT).; this is as expected. For one location, gpi = 810025, there are some significant differences from 2002 onwards; it is unclear where this originates from and this will be investigated further in the full PQAR for the product.
Figure 16: Daily image of the difference in soil moisture for the COMBINED products of C3S v201812 and ESA CCI SM v04.5 (C3S minus CCI).
The difference between the soil moisture in the C3S and CCI datasets (COMBINED) are shown in Figure 16 for the date 2018-06-21. This shows that the use of the different ASCAT data streams do have an effect on the final product, which is as expected. The key areas seem to be in high vegetation areas (although not consistently). This will be further investigated in the full PQAR for the product.
References
(JCGM), J. C. F. G. I. M. 2008. International vocabulary of metrology — Basic and general concepts and associated terms (VIM). VIM3: International Vocabulary of Metrology, 3, 104.
ALBERGEL, C., DORIGO, W., BALSAMO, G., MUNOZ-SABATER, J., DE ROSNAY, P., ISAKSEN, L., BROCCA, L., DE JEU, R. & WAGNER, W. 2013. Monitoring multi-decadal satellite earth observation of soil moisture products through land surface reanalyses. Remote Sensing of Environment, 138, 77-89.
ALBERGEL, C., RÜDIGER, C., CARRER, D., CALVET, J.-C., FRITZ, N., NAEIMI, V., BARTALIS, Z. & HASENAUER, S. 2008. An evaluation of ASCAT surface soil moisture products with in-situ observations in southwestern France. Hydrology and Earth System Sciences Discussions, 5, 2221-2250.
BALSAMO, G., ALBERGEL, C., BELJAARS, A., BOUSSETTA, S., BRUN, E., CLOKE, H., DEE, D., DUTRA, E., MUNÕZ-SABATER, J., PAPPENBERGER, F., DE ROSNAY, P., STOCKDALE, T. & VITART, F. 2015. ERA-Interim/Land: A global land surface reanalysis data set. Hydrology and Earth System Sciences, 19, 389-407.
BALSAMO, G., ALBERGEL, C., BELJAARS, A., BOUSSETTA, S., BRUN, E., CLOKE, H., DEE, D., DUTRA, E., PAPPENBERGER, F., DE ROSNAY, P., MUÑOZ-SABATER, J., STOCKDALE, F. & VITART, F. 2012. ERA-Interim/Land: A global land-surface reanalysis based on ERA-Interim meteorological forcing.
BROCCA, L., HASENAUER, S., LACAVA, T., MELONE, F., MORAMARCO, T., WAGNER, W., DORIGO, W., MATGEN, P., MARTÍNEZ-FERNÁNDEZ, J., LLORENS, P., LATRON, J., MARTIN, C. & BITTELLI, M. 2011. Soil moisture estimation through ASCAT and AMSR-E sensors: An intercomparison and validation study across Europe. Remote Sensing of Environment, 115, 3390-3408.
CHUNG, D., DORIGO, W., R.A.M DE JEU., HAHN, S., Melzer, T., PARINUSSA, R. M., PAULIK, C., REIMER, C., VREUGDENHIL, M., BARRETT, B., WAGNER, W., KIDD, R., GRUBER, A., SCANLON, T. 2018. Algorithm Theoretical Baseline Document (ATBD) D2.1 Version 4.4: Executive Summary.
CROW, W. T., BERG, A. A., COSH, M. H., LOEW, A., MOHANTY, B. P., PANCIERA, R., DE ROSNAY, P., RYU, D. & WALKER, J. P. 2012. Upscaling Sparse Ground-Based Soil Moisture Observations For The Validation Of Coarse-Resolution Satellite Soil Moisture Products. Reviews of Geophysics, 50, 1-20.
DEE, D. P., UPPALA, S. M., SIMMONS, A. J., BERRISFORD, P., POLI, P., KOBAYASHI, S., ANDRAE, U., BALMASEDA, M. A., BALSAMO, G., BAUER, P., BECHTOLD, P., BELJAARS, A. C. M., VAN DE BERG, L., BIDLOT, J., BORMANN, N., DELSOL, C., DRAGANI, R., FUENTES, M., GEER, A. J., HAIMBERGER, L., HEALY, S. B., HERSBACH, H., HÓLM, E. V., ISAKSEN, L., KÅLLBERG, P., KÖHLER, M., MATRICARDI, M., MCNALLY, A. P., MONGE-SANZ, B. M., MORCRETTE, J. J., PARK, B. K., PEUBEY, C., DE ROSNAY, P., TAVOLATO, C., THÉPAUT, J. N. & VITART, F. 2011. The ERA-Interim reanalysis: Configuration and performance of the data assimilation system. Quarterly Journal of the Royal Meteorological Society, 137, 553-597.
DORIGO, W., WAGNER, W., ALBERGEL, C., ALBRECHT, F., BALSAMO, G., BROCCA, L., CHUNG, D., ERTL, M., FORKEL, M., GRUBER, A., HAAS, E., HAMER, P. D., HIRSCHI, M., IKONEN, J., DE JEU, R., KIDD, R., LAHOZ, W., LIU, Y. Y., MIRALLES, D., MISTELBAUER, T., NICOLAI-SHAW, N., PARINUSSA, R., PRATOLA, C., REIMER, C., VAN DER SCHALIE, R., SENEVIRATNE, S. I., SMOLANDER, T. & LECOMTE, P. 2017. ESA CCI Soil Moisture for improved Earth system understanding: State-of-the art and future directions. Remote Sensing of Environment.
DORIGO, W. A., GRUBER, A., DE JEU, R. A. M., WAGNER, W., STACKE, T., LOEW, A., ALBERGEL, C., BROCCA, L., CHUNG, D., PARINUSSA, R. M. & KIDD, R. 2015. Evaluation of the ESA CCI soil moisture product using ground-based observations. Remote Sensing of Environment, 162, 380-395.
DORIGO, W. A., XAVER, A., VREUGDENHIL, M., GRUBER, A., HEGYIOVÁ, A., SANCHIS-DUFAU, A. D., ZAMOJSKI, D., CORDES, C., WAGNER, W. & DRUSCH, M. 2013. Global Automated Quality Control of In Situ Soil Moisture Data from the International Soil Moisture Network. Vadose Zone Journal, 12, 0.
DORIGO, W., VAN OEVELEN, P., WAGNER, W., DRUSCH, M., MECKLENBURG, S., ROBOCK, A. & JACKSON, T. 2011. A new international network for in situ soil moisture data. Eos, 92, 141-142.
ENTEKHABI, D., REICHLE, R. H., KOSTER, R. D. & CROW, W. T. 2010. Performance Metrics for Soil Moisture Retrievals and Application Requirements. Journal of Hydrometeorology, 11, 832-840.
GRUBER, A., DORIGO, W., CROW, W. & WAGNER, W. 2017. Triple Collocation-Based Merging of Satellite Soil Moisture Retrievals. IEEE Transactions of Geoscience and Remote Sensing.
GRUBER, A., SU, C.-H., ZWIEBACK, S., CROW, W., DORIGO, W. & WAGNER, W. 2016. Recent advances in (soil moisture) triple collocation analysis. International Journal of Applied Earth Observation and Geoinformation, 45, 200-211.
GRUBER, A., DORIGO, W. A., ZWIEBACK, S., XAVER, A. & WAGNER, W. 2013. Characterizing Coarse-Scale Representativeness of in situ Soil Moisture Measurements from the International Soil Moisture Network. Vadose Zone Journal, 12, 0.
GRUBER, A., SCANLON, T., VAN DER SCHALIE, R., WAGNER, W., AND DORIGO, W. 2019: Evolution of the ESA CCI Soil Moisture climate data records and their underlying merging methodology, Earth Syst. Sci. Data, 11, 717–739, https://doi.org/10.5194/essd-11-717-2019
.
GRUHIER, C., DE ROSNAY, P., HASENAUER, S., HOLMES, T., DE JEU, R., KERR, Y., MOUGIN, E., NJOKU, E., TIMOUK, F., WAGNER, W. & ZRIBI, M. 2009. Soil moisture active and passive microwave products: intercomparison and evaluation over a Sahelian site. Hydrology and Earth System Sciences Discussions, 6, 5303–5339.
H. HERSBACH, P. DE ROSNAY, B. BELL, D. SCHEPERS, A. SIMMONS, C. SOCI, S. ABDALLA, M. ALONSO-BALMASEDA, G. BALSAMO, P. BECHTOLD, P. BERRISFORD, J.-R. BIDLOT, E. DE BOISS´ESON, M. BONAVITA, P. BROWNE, R. BUIZZA, P. DAHLGREN, D. DEE, R. DRAGANI, M. DIAMANTAKIS, J. FLEMMING, R. FORBES, A. J. GEER, T. HAIDEN, E. H´OLM, L. HAIMBERGER, R. HOGAN, A. HOR´ANYI, M. JANISKOVA, P. LALOYAUX, P. LOPEZ, J. MUNOZ-SABATER, C. PEUBEY, R. RADU, D. RICHARDSON, J. TH´EPAUT, F. VITART, X. YANG, E. ZS´OT´ER, AND H. ZUO 2018. Operational global reanalysis: Progress, future directions and synergies with NWP, ECMWF, ERA Report Series 27, Dec. 2018. doi: 10.21957/tkic6g3wm.
HILLEL, D. 2004. Introduction to environmental soil physics, Elsevier Academic Press.
HOLMES, T. R. H., DE JEU, R. A. M., OWE, M. & DOLMAN, A. J. 2009. Land surface temperature from Ka band (37 GHz) passive microwave observations. Journal of Geophysical Research-Atmospheres, 114.
INSTITUTION, B. S. 2015. BSI Standards Publication BS EN ISO 9000:2015 Quality management systems Fundamentals and vocabulary.
JUSTICE, C., BELWARD, A., MORISETTE, J., LEWIS, P., PRIVETTE, J. & BARET, F. 2000. Developments in the 'validation' of satellite sensor products for the study of the land surface. 21, 3383-3390.
LIU, Y. Y., DORIGO, W. A., PARINUSSA, R. M., DE JEU, R. A. M., WAGNER, W., MCCABE, M. F., EVANS, J. P. & VAN DIJK, A. I. J. M. 2012. Trend-preserving blending of passive and active microwave soil moisture retrievals. Remote Sensing of Environment, 123, 280-297.
LOEW, A., BELL, W., BROCCA, L., BULGIN, C. E., BURDANOWITZ, J., CALBET, X., DONNER, R. V., GHENT, D., GRUBER, A., KAMINSKI, T., KINZEL, J., KLEPP, C., LAMBERT, J. C., SCHAEPMAN-STRUB, G., SCHRODER, M. & VERHOELST, T. 2017. Validation practices for satellite-based Earth observation data across communities. Reviews of Geophysics, 55, 779-817.
MITTELBACH, H., SENEVIRATNE, S. I., HIRSCHI, M., DORIGO, W. A., WAGNER, W., PARINUSSA, R. M., SCARROTT, R., DWYER, N., HAAS, E., RAUTIANINEN, K. & LAHOZ, W. A. 2012. ESA CCI: Soil Moisture: Product Validation Plan.
MITTELBACH, H., HIRSCHI, M., PRATOLA, C., SMOLANDER, T., LAHOZ, W. A., DWYER, N., RAUTIANINEN, K. & SENEVIRANTNE, S. I. 2014. ESA CCI Soil Moisture: Product Validation and Intercomparison Report (PVIR).
NAEIMI, V., PAULIK, C., BARTSCH, A., WAGNER, W., KIDD, R., PARK, S. E., ELGER, K. & BOIKE, J. 2012. ASCAT Surface State Flag (SSF): Extracting Information on Surface Freeze/Thaw Conditions From Backscatter Data Using an Empirical Threshold-Analysis Algorithm. Ieee Transactions on Geoscience and Remote Sensing, 50, 2566-2582.
NICOLAI-SHAW, N., HIRSCHI, M., MITTELBACH, H. & SENEVIRATNE, S. I. 2015. Spatial representativeness of soil moisture using in situ , remote sensing , and land reanalysis data.
PARINUSSA, R., DE JEU, R., VAN DER SCHALIE, R., CROW, W., LEI, F. & HOLMES, T. 2016. A Quasi-Global Approach to Improve Day-Time Satellite Surface Soil Moisture Anomalies through the Land Surface Temperature Input. Climate, 4.
PAULIK, C. & REIMER, C. 2018. C3S System Quality Assurance Document (SQAD): Soil Moisture (v201807)
PLUMMER, S., LECOMTE, P. & DOHERTY, M. 2017. The ESA Climate Change Initiative (CCI): A European contribution to the generation of the Global Climate Observing System. Remote Sensing of Environment.
PREIMESBERGER, W., SU, C. H., GRUBER, A. & DORIGO, W. Homogeneity testing of the global ESA CCI multi-satellite soil moisture climate data record. EGU, 2017. Vienna, Austria.
RÜDIGER, C., CALVET, J.-C., GRUHIER, C., HOLMES, T. R. H., DE JEU, R. A. M. & WAGNER, W. 2009. An Intercomparison of ERS-Scat and AMSR-E Soil Moisture Observations with Model Simulations over France. Journal of Hydrometeorology, 10, 431-447.
STEELE-DUNNE, S. C., FRIESEN, J. & VAN DE GIESEN, N. 2012. Using Diurnal Variation in Backscatter to Detect Vegetation Water Stress. Ieee Transactions on Geoscience and Remote Sensing, 50, 2618-2629.
SU, C. H., RYU, D., DORIGO, W., ZWIEBACK, S., GRUBER, A., ALBERGEL, C., REICHLE, R. H. & WAGNER, W. 2016. Homogeneity of a global multisatellite soil moisture climate data record. Geophysical Research Letters, 43, 11,245–11,252.
WAGNER, W., HAHN, S., KIDD, R., MELZER, T., BARTALIS, Z., HASENAUER, S., FIGA-SALDANA, J., DE ROSNAY, P., JANN, A., SCHNEIDER, S., KOMMA, J., KUBU, G., BRUGGER, K., AUBRECHT, C., ZUGER, J., GANGKOFNER, U., KIENBERGER, S., BROCCA, L., WANG, Y., BLOSCHL, G., EITZINGER, J., STEINNOCHER, K., ZEIL, P. & RUBEL, F. 2013. The ASCAT Soil Moisture Product: A Review of its Specifications, Validation Results, and Emerging Applications. Meteorologische Zeitschrift, 22, 5-33.
WMO 2016. The Global Observing System for Climate (GCOS): Implementation Needs.
ZWIEBACK, S., SCIPAL, K., DORIGO, W. & WAGNER, W. 2012. Structural and statistical properties of the collocation technique for error characterization. Nonlinear Processes in Geophysics, 19, 69–80.
