Contributors: W. Preimesberger (WP, TU Wien), W. Dorigo (WD, TU Wien), D. Aberer (DA, TU Wien), A. Dostalova (AD, EODC), R. Kidd (RK, EODC), T. Frederikse (TF, Planet Labs / Vandersat) 

Issued by: EODC GmbH/Alena Dostalova

Date: 28/08/2024

Ref: C3S2_312a_Lot4.WP2-FDDP-SM-v2_202312_SM_PQAR-v5_i1.2

Official reference number service contract: 2021/C3S2_312a_Lot4_EODC/SC1

Table of Contents

History of modifications

Version

Date

Description of modification

Chapters / Sections

i0.1

17/05/2024

All sections were updated for CDR v5.0 (public version v202312)

All

i1.0

31/05/2024

Addition of Annex D, internal review and document finalization

All

i1.1

08/07/2024

External independent review and document finalization

All

i1.2

28/08/2024

Updated Figure 55

Annex D

List of datasets covered by this document

Deliverable ID

Product Title

Product type (CDR, ICDR)

C3S version number

Product ID

WP2-FDDP-SM-CDR-v5

Surface Soil Moisture (Passive) Daily

CDR

v5.0

v202312

WP2-FDDP-SM-CDR-v5

Surface Soil Moisture (Passive) Dekadal

CDR

v5.0

v202312

WP2-FDDP-SM-CDR-v5

Surface Soil Moisture (Passive) Monthly

CDR

v5.0

v202312

WP2-FDDP-SM-CDR-v5

Surface Soil Moisture (Active) Daily

CDR

v5.0

v202312

WP2-FDDP-SM-CDR-v5

Surface Soil Moisture (Active) Dekadal

CDR

v5.0

v202312

WP2-FDDP-SM-CDR-v5

Surface Soil Moisture (Active) Monthly

CDR

v5.0

v202312

WP2-FDDP-SM-CDR-v5

Surface Soil Moisture (Combined) Daily

CDR

v5.0

v202312

WP2-FDDP-SM-CDR-v5

Surface Soil Moisture (Combined) Dekadal

CDR

v5.0

v202312

WP2-FDDP-SM-CDR-v5

Surface Soil Moisture (Combined) Monthly

CDR

v5.0

v202312

Related documents 

Reference ID

Document

RD1

Preimesberger W. et al. (2024). C3S Soil Moisture Version v202312: Product User Guide and Specification. Document ref: C3S2_312a_Lot4.WP2-FDDP-SM-v2_202312_SM_PUGS-v5_i1.1, Available at SM v202312: Product User Guide and Specification (PUGS) (resource validated 16th May 2024)

RD2

Preimesberger W. et al. (2024) C3S Soil Moisture Version v202312: Product Quality Assurance Document. Document ref: C3S2_312a_Lot4.WP1-PDDP-SM-v2_202312_SM_PQAD-v5_i1.1, Available at: SM v202312: Product Quality Assurance Document (PQAD) (resource validated 16th May 2024)

RD3

De Jeu R. et al. (2024) C3S Soil Moisture Version v202312: Algorithm Theoretical Basis Document. Document ref: C3S2_312a_Lot4.WP2-FDDP-SM-v2_202312_SM_ATBD-v5_i1.1

RD4

Preimesberger W. et al. (2024) C3S Soil Moisture Version v202312: Target Requirements and Gap Analysis Document. Document ref: C3S2_312a_Lot4.WP3-TRGAD-SM-v2_202304_SM_TR_GA_i1.1, Available at: Soil Moisture 2023: Target Requirements and Gap Analysis Document (TRGAD) (resource validated 16th May 2024)

RD5

Global Climate Observing System (2016) THE GLOBAL OBSERVING SYSTEM FOR CLIMATE: IMPLEMENTATION NEEDS, GCOS-200, https://library.wmo.int/doc_num.php?explnum_id=3417 (resource validated 16th May 2024)

RD6

Global Climate Observing System (2022), The 2022 GCOS ECVs Requirements, GCOS-245, https://library.wmo.int/doc_num.php?explnum_id=11318 (resource validated 16th May 2024)

RD7

Preimesberger W. et al. (2023). C3S Soil Moisture Version v202212: Product Quality Assessment Report. Document Ref. C3S2_312a_Lot4.WP2-FDDP-SM-v1_202212_SM_PQAR-v4_i1.2, Available at: SM v202212: Product Quality Assessment Report (PQAR) (resource validated 16th May 2024)

RD8


Hirschi, M., Stradiotti, P., Preimesberger, W., Dorigo, W., & Kidd, R. (2023). ESA Climate Change Initiative Plus - Soil Moisture Product Validation and Intercomparison Report (PVIR) Supporting Product version v08.1 (issue 1.0). Zenodo. https://doi.org/10.5281/zenodo.8320930

Acronyms

Acronym

Definition

AMI-WS

Active Microwave Instrument – Wind Scatterometer (ERS-1 & 2)

AMSR2

Advanced Microwave Scanning Radiometer 2

AMSR-E

Advanced Microwave Scanning Radiometer-Earth Observing System

ASCAT

Advanced Scatterometer (Metop)

ATBD

Algorithm Theoretical Basis Document

BfG

Federal Institute of Hydrology

C3S

Copernicus Climate Change Service

CCI

Climate Change Initiative

CDR

Climate Data Record

CDS

Climate Data Store

CEOS

Committee in Earth Observation Satellites

CF

Climate Forecast

CMUG

Climate Modelling User Group

ECV

Essential Climate Variable

ECMWF

European Centre for Medium Range Weather Forecasting

EODC

Earth Observation Data Centre GmbH

ERA

ECMWF Reanalysis

ERS

European Remote-Sensing Satellite

ESA

European Space Agency

FRM

Fiducial Reference Measurements

FRM4SM

Fiducial Reference Measurements for Soil Moisture

FY

FengYun

GCOS

Global Climate Observing System

GEO

Group on Earth Observation

GHRSST

Group for High Resolution Sea Surface Temperature

GLDAS

Global Land Data Assimilation System

GPI

Grid point index (identifier for unique lon / lat combination)

GPM

Global Precipitation Measurement

H-SAF

Hydrological Satellite Application Facility (EUMETSAT)

HWSD

Harmonised World Soil Database

ICDR

Intermediate Climate Data Record

ISMN

International Soil Moisture Network

IQR

Interquartile Range

KPI

Key Performance Indicator

LSM

Land Surface Model

LPRM

Land Parameter Retrieval Model

MetOp

Meteorological Operational satellite

NetCDF

Network Common Data Format

NRT

Near Real Time

PQAD

Product Quality Assurance Document

PQAR

Product Quality Assessment Report

PUGS

Product User Guide and Specification

QA

Quality Assurance

QA4EO

Quality Assurance framework for Earth Observation

QA4SM

Quality Assurance for Soil Moisture

RFI

Radio Frequency Interference

RZSM

Root Zone Soil Moisture

SM

Soil Moisture

SMMR

Scanning Multichannel Microwave Radiometer

SMAP

Soil Moisture Active Passive

SMOS

Soil Moisture and Ocean Salinity

SNR

Signal-to-Noise Ratio

SSM

Surface Soil Moisture

SSM/I

Special Sensor Microwave Imager

TCA

Triple Collocation Analysis

TMI

TRMM Microwave Imager

TRGAD

Target Requirements and Gap Analysis Document

TRMM

Tropical Rainfall Measuring Mission

TU Wien

Vienna University of Technology

ubRMSD

unbiased Root Mean Square Difference

UNFCCC

United Nations Framework Convention on Climate Change

US

United States

VOD

Vegetation Optical Depth

WGS

World Geodetic System

WindSat

WindSat Spaceborne Polarimetric Microwave Radiometer

General definitions

Active (soil moisture) retrieval: the process of modeling soil moisture from radar (scatterometer and synthetic aperture radar) measurements. The measurand of active microwave remote sensing systems is called “backscatter”.

Accuracy: The closeness of agreement between a measured quantity value and a true quantity value of a measurand ((JCGM), 2008). The metrics used here to represent accuracy are correlation and unbiased Root Mean Square Difference (ubRMSD). These metrics are commonly used throughout the scientific community as measures of accuracy (Entekhabi et al., 2010).

Backscatter is the measurand of “active” microwave remote sensing systems (radar). As the energy pulses emitted by the radar hit the surface, a scattering effect occurs and part of the energy is reflected. The received energy is called “backscatter”, with rough surfaces producing stronger signals than smooth surfaces. It comprises reflections from the soil surface layer (“surface scatter”), vegetation (“volume scatter”), and interactions of the two. Under very dry soil conditions, structural features in deeper soil layers can act as volume scatterers (“subsurface scattering”).

Bias : “Bias is defined as an estimate of the systematic measurement error. (GCOS-200) [RD5]

Break detection and correction: Additional processing step introduced in the Combined Soil Moisture product version v202312. The methods detect and correct inhomogeneties in the merged satellite records (breaks) that may occur as a result of merging different sensor combinations over time. More information is available in the product Algorithm Theoretical Basis Document (ATBD) [RD3].

Breakthrough requirement: An Essential Climate Variable (ECV) requirement level set by Global Climate Observing System (GCOS) which “[…] if achieved, would result in a significant improvement for the targeted application […] at which specified uses within climate monitoring become possible. It may be appropriate to have different breakthrough values for different uses.” (GCOS-245) [RD6]

Brightness Temperature is the measurand of “passive“ microwave remote sensing systems (radiometers). Brightness temperature (in degrees Kelvin) is a function of kinetic temperature and emissivity. Wet soils have a higher emissivity than dry soils and therefore a higher brightness temperature. Passive soil moisture retrieval uses this difference between kinetic temperature and brightness temperature, to model the amount of water available in the soil of the observed area while taking into account factors such as the water held by vegetation.

Dekad: the period or interval of 10 days

Error: “The term error refers to the deviation of a single measurement (estimate) from the true value of the quantity being measured (estimated), which is always unknown” (Gruber et al., 2020)

Fiducial Reference Measurements (FRM):  Suite of independent, fully characterized, and traceable ground measurements that follow the guidelines outlined by the Group on Earth Observation (GEO) / Committee in Earth Observation Satellites (CEOS) Quality Assurance framework for Earth Observation (QA4EO)1.

Key Performance Indicators (KPIs): A set of performance measures designed to rate the quality of satellite soil moisture observations. Based on suggestions from the Global Climate Observing System (GCOS), the Climate Modelling User Group (CMUG), and other community-agreed standards.

Koeppen-Geiger Climate Classification: Global classification of regions based on their climates. Contains 5 main classes with multiple sub-classes: A (tropical), B (arid), C (temperate), D (continental), and E (polar) climates. In the context of Copernicus Climate Change Service (C3S) soil moisture, the classification of Peel et al. (2007) is used.

Level 2 pre-processed (L2P): this is a designation of satellite data processing level. “Level 2” means geophysical variables derived from Level 1 source data on the same grid (typically the satellite swath projection). “Pre-processed” means ancillary data and metadata added following Group for High-Resolution Sea Surface Temperature (GHRSST) Data Specification.

Level 3 /uncollated/collated/super-collated (L3U/L3C/L3S): this is a designation of satellite data processing level. “Level 3” indicates that the satellite data is a geophysical quantity (retrieval) that has been averaged where data are available to a regular grid in time and space. “Uncollated” means L2 data granules have been remapped to a regular latitude/longitude grid without combining observations from multiple source files. L3U files will typically be “sparse” corresponding to a single satellite orbit. “Collated” means observations from multiple images/orbits from a single instrument combined into a space-time grid. A typical L3C file may contain all the observations from a single instrument in a 24-hour period. “Super-collated” indicates that (for those periods where more than one satellite data stream delivering the geophysical quantity has been available) the data from more than one satellite have been gridded together into a single grid-cell estimate, where relevant.

Passive (soil moisture) retrieval: the process of modeling soil moisture from radiometer measurements. The measurand of passive microwave remote sensing is called “brightness temperature”). The retrieval model in the context of C3S soil moisture is generally the Land Parameter Retrieval Model.

Radiometer: Spaceborne radiometers are satellite-carried sensors that measure energy in the microwave domain emitted by the Earth. The amount of radiation emitted by an object in the microwave domain (~1-20 GHz). The observed quantity is called “brightness temperature” and depends on the kinetic temperature of an object and its emissivity. Due to the high emissivity of water compared to dry matter, radiometer measurements of the Earth’s surface contain information on the water content in the observed area.

Scatterometer: Spaceborne scatterometers are satellite-carried sensors that use microwave radars to measure the reflection or scattering effect produced by scanning a large area on the surface of the Earth. The initially submitted pulses of energy are reflected by the Earth’s surface depending on its geometrical and geophysical properties in the target area. The received energy is called “backscatter”. Soil moisture retrieval relies on the fact that wet soils have a higher reflectivity (and therefore backscatter) than dry soils due to the high dielectric constant of liquid water compared to dry matter.

Stability: “The change in bias over time” (GCOS-245) [RD6]. “Stability may be thought of as the extent to which the uncertainty of measurement remains constant with time. […] ‘Stability’ refer[s] to the maximum acceptable change in systematic error, usually per decade.” (GCOS-200) [RD5]

Target requirement: ideal requirement which would result in a considerable improvement for the target application.

Threshold requirement: minimum requirement to be met to ensure data are useful.

Uncertainty: “Satellite soil moisture retrievals […] usually contain considerable systematic errors which, especially for model calibration and refinement, provide better insight when estimated separately from random errors. Therefore, we use the term bias to refer to systematic errors only and the term uncertainty to refer to random errors only, specifically to their standard deviation (or variance)” (Gruber et al., 2020)

1 https://qa4eo.org/ resource validated 16th May 2024

Scope of the document

The purpose of this document is to describe the results of the Quality Assessment (QA) for the soil moisture product developed by the Vienna University of Technology (TU Wien), Earth Observation Data Centre (EODC), and Planet Labs/VanderSat for the Copernicus Climate Change Service (C3S) hosted by the European Centre for Medium Range Weather Forecasting (ECMWF). The product version assessed in this report is v202312.0.0, which was produced in March 2024.

Executive summary

The C3S soil moisture product suite provides PASSIVE, ACTIVE, and COMBINED (passive + active) microwave soil moisture products on a daily, dekadal (10-days), and monthly basis. The data are provided in a regular 0.25-degree grid based on the World Geodetic System 1984 (WGS 84) reference system. The product is available globally between November 1978 and present-day (for PASSIVE and COMBINED) and between 1991 and present-day (for ACTIVE). For details about the products, we refer to the Product User Guide and Specification (PUGS) [RD1].

This document presents the results of QA activities that have been undertaken for the current Climate Data Record (CDR) product version v202312.0.0. The Intermediate Climate Data Records (ICDR) are currently not assessed. However, note that to achieve maximum consistency between CDR and ICDR, both products use the same Level 2 products (based on Near Real Time (NRT) data streams) and merging algorithms and thus have very similar quality characteristics.

Chapter 1 summarises the quality assessment methodology, described fully in the Product Quality Assurance Document (PQAD) [RD2]. The results described here are primarily for the COMBINED daily product, however, most assessments were also undertaken for the ACTIVE and PASSIVE products.

Chapter 2 contains QA results for (i) accuracy assessment, (ii) stability assessment, (iii) demonstration of uncertainty estimates, (iv) comparison to previous versions of the product, and (v) completeness/consistency assessment. The first two sections focus on demonstrating that the Key Performance Indicators (KPIs) set for the product are met. Note these KPIs take into account the Global Climate Observing System (GCOS) and user requirements for the product described in the Target Requirements and Gap Analysis Document (TRGAD) [RD4 ]. 

Accuracy Assessment: In general, there is a slight variability in the correlation between the three C3S SM products, with correlations ranging from 0.4 to above 0.8; depending on the conditions and the locations of the in situ stations used. The unbiased Root Mean Square Difference (ubRMSD), which can be directly taken as a measure of accuracy, is demonstrated to be below 0.10 m/ m3 for all the different conditions analyzed and often below the GCOS target of 0.04 m/ m3. Therefore, the KPIs for accuracy have been met for in situ observations. The global comparison against the ERA5-Land (ECMWF Reanalysis v5) has shown that the KPI threshold is met in most areas when ERA5-Land is used as a reference for the satellite data. Notable exceptions are some areas with reduced data coverage in the sub-Arctic zone.

Stability Assessment: The stability of the C3S product has been assessed in terms of the change in accuracy (when compared to International Soil Moisture Network (ISMN) network measurements). The accuracy between the products (ubRMSD) has been calculated per year, as well as trends in the median yearly accuracy. The KPI threshold for stability of 0.05 m³ / m³ / y is met when assessed using this method for all tested locations. However, the stability assessment algorithms applied here are preliminary and not a community-approved standard best practice.

Trend Assessment: Long-term trends are contradictory between the three products, indicating intercalibration is still insufficient and/or affected by systematic differences between products (e.g. temporal coverage).

Uncertainty Assessment: Intra-annual signal-to-noise ratio estimates for all sensors indicates different performance levels depending on the sensor type/frequency band, land cover and seasonal effects (e.g, vegetation dynamics).

Break detection assessment: Temporal inconsistencies in the COMBINED product were detected using statistical tests and successfully removed as part of the C3S SM processing chain.

A comparison to previous products has been provided for the daily, monthly, and dekadal products separately. The assessment demonstrates that the correlation with ISMN and ERA5-Land is similar to the previous version.

The main algorithmic updates in v202312 are:

  • The uncertainty estimates obtained through Triple Collocation Analysis (TCA) and the consequent merging weights are now generated on a seasonal basis (intra-annually). The product uncertainty field is now more representative of actual seasonal uncertainties in soil moisture retrievals.
  • The barren ground flagging strategy (as well as the flagging of frozen soils) has been changed to a majority decision, where at least 50 % + 1 of the observing sensors are needed to raise the flag. The barren-ground flag is kept optional, i.e., the soil moisture data is not masked out in this case.
  • A method to detect and correct temporal inhomogeneities ("breaks") in the COMBINED product was applied (Preimesberger et al., 2021)

The spatial and temporal coverage of the product is presented in terms of the number of available observations. It is shown that the coverage is better in Europe, southern Africa, Australia, and the eastern United States (US) compared to the western US, Canada, and northern and central parts of Asia.  A significant decrease in data coverage is found, in particular for the ACTIVE product, but also affecting the COMBINED product . This is however in line with the underlying ESA CCI SM version and therefore accepted as an algorithmic change rather than a processing error.

Furthermore, a detailed assessment of the product has been undertaken, in particular for the ACTIVE and PASSIVE products as well as a detailed comparison against the previous product version. These assessments show that the dataset was generated correctly.

However, some (previously known) issues remain in the current version:

  • ACTIVE wetting trends: Artificial wetting trends in Hydrological Satellite Application Facility (H-SAF) ASCAT Surface Soil Moisture (SSM) also affect the ACTIVE product of C3S Soil Moisture (SM) after 2007 (and, to some extent, the COMBINED product). These trends are already present in ASCAT observed backscatter time series. They are caused by changes in land cover that are not sufficiently accounted for during the soil moisture retrieval process and therefore translate into a gradual wetting bias in the data. The trends are currently corrected in an experimental version of ASCAT SSM. This issue will be corrected once the change correction is also introduced to the operational H-SAF SSM products used in C3S SM.
  • Inconsistent long-term SM trends: Inconsistent long-term trends (1991 – 2020) are found between the ACTIVE, PASSIVE, and COMBINED products. While this can be partly related to differences in time range, data density, and scaling references of the products, it still reveals that there is a need to improve the long-term consistency of the product.

1. Product validation methodology

1.1. Validated products

This current document is applicable to the QA activities performed on the version of the CDR v202312.0.0 (produced in March 2024 ).

C3S soil moisture provides passive (named PASSIVE), active (ACTIVE), and passive + active (COMBINED) soil moisture CDRs on a daily, dekadal (10-days), and monthly basis (9 records for each version). The periods, over which each sensor is used, are provided in Figure 1 . The data are provided in a regular 0.25-degree grid based on the WGS 84 reference system. The product is available globally between November 1978 and present-day (for PASSIVE and COMBINED) and between 1991 and present-day (for ACTIVE). The product has been produced by TU Wien, Planet Labs / VanderSat, and EODC. The ASCAT SSM product mentioned in some parts of this report is developed and provided by H-SAF.

Figure  1: Sensors and merging periods for the C3S soil moisture product (ACTIVE, PASSIVE, and COMBINED) version v202312. Letters beside the sensor/satellite names indicate the observation frequency band(s).

The C3S soil moisture product is generated from a set of passive microwave radiometers and active microwave scatterometers (red and blue sensors respectively in Figure 1 ). 

Radiometers (passive) include the Scanning Multichannel Microwave Radiometer (SMMR), Special Sensor Microwave Imager (SSM/I), Tropical Rainfall Measuring Mission (TRMM) Microwave Imager (TMI), Advanced Microwave Scanning Radiometer-Earth Observing System (AMSR-E), WindSat Spaceborne Polarimetric Microwave Radiometer (WindSat), Advanced Microwave Scanning Radiometer 2 (AMSR2), Soil Moisture and Ocean Salinity (SMOS), Soil Moisture Active Passive (SMAP), FengYun (FY) 3B/3C/3D and Global Precipitation Measurement (GPM) satellites. 

Scatterometer (active) observations are collected by the Active Microwave Instrument - Wind Scatterometer (AMI-WS) (onboard European Remote-Sensing Satellites (ERS)-1 and 2) and ASCAT (onboard Meteorological Operational (MetOp) satellites MetOp-A, MetOp-B and MetOp-C) sensors.

The “ACTIVE product” and the “PASSIVE product” are created by fusing scatterometer and radiometer Level 2 soil moisture products respectively; the “COMBINED product” is created by fusing Level 2 soil moisture products from both sensor types. Data files are provided in NetCDF-4 format as daily, dekadal, and monthly images and comply with CF-1.8 conventions2.

A detailed description of the product generation of C3S v202312.0.0 is provided in the Algorithm Theoretical Basis Document (ATBD) [RD3] with further information on the product given in the PUGS [ RD1 ]. The underlying algorithm is based on that used in the generation of the publicly released European Space Agency (ESA) Climate Change Initiative (CCI) version 8.1 which is described in in  Plummer et al. (2017), Wagner et al. (2012), Liu et al. (2012), Dorigo et al. (2017), Gruber et al. (2019) and [RD8]. In addition, detailed provenance traceability information can be found in the metadata of the product.

2 CF conventions: www.cfconventions.org (resource validated 16th May 2024)

1.2. Description of reference datasets

A combination of in situ and global reference datasets is utilized to assess the quality of the C3S soil moisture product. The utilized datasets are briefly described in this section with further details provided in the PQAD [RD2].

Compared to the previous version of this report, Global Land Data Assimilation System (GLDAS) Noah and ERA5 soil moisture products are no longer used as reference data sets for the validation of C3S SM. GLDAS is used in the generation of the COMBINED product and therefore not independent. ERA5-Land soil moisture fields are generally more accurate than ERA5 (Muñoz-Sabater et al. 2021) .

1.2.1. International Soil Moisture Network (ISMN)

This ISMN3 is a database hosted by the German Federal Institute of Hydrology (BfG). It is a global collocation of available in situ soil moisture measurements from operational networks and validation campaigns. In situ data are collected from the individual network providers, harmonized, and made available to users (Dorigo et al. 2021). The data within the ISMN are subject to quality controls and provided with quality flags (Dorigo et al. 2013). The quality controls include an assessment against a possible range of important metrological variables which are applied equally to all datasets. The yearly temporal coverage and time series length vary largely between stations. Often there are multiple soil moisture sensors present at a station in similar depths.
A snapshot of the full ISMN archive was downloaded in March 2024. This snapshot contains 4027 soil moisture time series from sensors in the top 10 cm of soil (Figure 2).


Figure 2: ISMN stations (as of March 2024) and sensor count. Stations with 1 (top left), 2 (top right), 3-6 (bottom left), and >7 (bottom right) soil moisture sensors in 0-10 cm depth are shown separately.

As part of ESA's Fiducial Reference Measurements for Soil Moisture (FRM4SM) project (resource validated 16th May 2024), a quality index for the "representativeness" of each in situ time series for the satellite data at radiometer scale (~25 km) was produced from the time series length and the Triple Collocation Analysis (TCA) based Signal-to-Noise Ratio (SNR). Sufficiently long time series with a high SNR (>3 dB) at 95% confidence are classified as "very representative" or "representative". Applying this quality indicator to all ISMN time series results in a subset of 1314 high-quality sensor time series. Their location is shown in Figure 3. The so-found subset forms the reference for the evaluation of the C3S SM.


Figure 3: Subset of the ISMN database to include "very representative" and "representative" measurements. Based on the FRM4SM quality indicator. Note that points in the graphic often overlap as most networks consist of stations that are spatially close together (clusters).

ISMN stations are distributed unevenly globally and often appear in clusters. Most measurements are taken within the continental United States and Europe. Validation results are therefore only representative of the environmental regimes covered by these stations (temperate climates) and mostly exclude (sub)equatorial and (sub)polar regions, highly organic soils as well as deserts.

3 ISMN website: https://ismn.earth/en/ (resource validated 16th May 2024)

1.2.1.1. Pre-processing

To calculate the metrics for each assessment, the settings summarized in Table 1 are used. The datasets are spatially and temporally matched, filtered for high-quality observations, and the C3S data are scaled to the ISMN data using mean/standard deviation scaling. The metrics are then calculated using all available observations in the period 1978-11-01 to 2023-12-31.

Table 1: Settings used in the assessment of the daily C3S soil moisture product against the ISMN

Setting

Details

Temporal Matching

The default approach is to use a temporal window of 1 hour to find the closest match between the satellite observation time stamp (at 0:00 UTC for C3S SM) and in situ or reanalysis datasets, i.e. the in reference data at 0:00 UTC is used. 

For some validation runs the daily mean of all compared products is used (this is indicated for each validation run when applied).

Spatial Matching

For all comparisons to in situ observations, the nearest C3S (and ERA5-Land) grid cell is found using the lon/lat of the ISMN station metadata. Only grid cells for which the central point is within a radius of <30 km around an in situ station are considered.

Scaling

The default approach is to remove additive and multiplicative biases between datasets due to spatial scale mismatch using a mean/standard deviation scaling approach (see Gruber et al., 2020). Correlation scores are not affected by the scaling step, ubRMSD values are usually slightly lower when computed from scaled compared to unscaled values.

Filters

The ISMN data have been filtered on the "soil moisture_flag" column such that only observations marked "G" are utilized5 (Dorigo et al. 2013).

The depths of the ISMN sensors used are indicated for each validation run. Usually, data from 0-10 cm depth - in some cases 0-5 cm - are used.

Anomalies

Anomalies are computed by subtracting the long-term average (climatology) from the measurements. The climatological reference period is from 1990 to 2020.

5 More information on the ISMN quality flags can be found at  https://ismn.earth/en/data/ismn-quality-flags/ (resource validated 16th May 2024)

1.2.2. ERA5-Land

ERA5-Land (Muñoz-Sabater et al. 2021), produced by ECMWF, is available from 1950 to present with a few days delay. It provides surface variables with an increased spatial resolution compared to ERA5 (Hersbach et al. 2020). Soil Moisture in ERA5-Land is available in 1-hour intervals on a ~9km resolution and without gaps. ERA5-Land provides various (land) variables, such as soil moisture or soil temperature. Both are available for four soil layers (0-7, 7-28, 28-100 and 100-289 cm). Here we use ERA5-Land data from 1981 to 2023 extracted for hours 0, 6, 12, and 18 of each day. Data for the top soil layer (0-7 cm) is used for comparison to the C3S SM satellite products.

1.2.3. ESA CCI SM

The CCI project was initiated in 2009 by ESA in response to the United Nations Framework Convention on Climate Change (UNFCCC) and GCOS needs for Essential Climate Variable (ECV) databases (Plummer et al. 2017). In 2012, ESA released the first multi-decadal, global satellite-observed soil moisture dataset, named ESA CCI SM, combining various single-sensor active and passive microwave soil moisture products (Dorigo et al. 2017).

The C3S product at version v202312 is based (scientifically, algorithmically) on version v8.1 of ESA CCI SM. There are however small differences in terms of integrated data streams due to the operational production of C3S ICDRs: C3S SM uses different ASCAT data streams that provide observations in NRT. Daytime observations are not used for any of the operational passive sensors as the current daytime retrieval algorithm is not compatible with operational products (ASMR2, SMOS, SMAP, GPM). The LPRM data used in ESA CCI SM and C3S SM is not exactly the same. C3S SM uses a reprocessed version of all sensors with some changes and bug fixes applied after the production of ESA CCI SM.

1.3. Description of product validation methodology

1.3.1. Method Overview

The methodology used in the assessment of the soil moisture product is described in the PQAD [RD2]. The methodology, including details on the validated products and the validating datasets, are briefly described here in the relevant sections. A summary of the validation results for the previous dataset version (v202212) may be found in the relevant PQAR document [RD7]. All box plots in the following chapters indicate the median, interquartile range, and 1.5*interquartile range of metrics found between the satellite and reference data products.

The quality assessment includes the following:

  1. Assessment of the spatial and temporal completeness of the products (Section 2.1)
  2. Accuracy assessment against in situ observations from the ISMN (Section 2.2),
  3. Accuracy assessment against ERA5-Land reanalysis (Section 2.3),
  4. Stability analysis through monitoring of accuracy trends and dataset statistics (Section 2.4),
  5. Analysis of time series at selected locations (Section 2.5)
  6. Analysis of the uncertainty information provided with the dataset (Section 2.6)
  7. Analysis of temporal consistency and the performance of temporal break correction (Section 2.7)

Additional information is also provided in the Annexes of the report:

  1. Annex A: The main body of the report considers mainly validation of the COMBINED product CDR. Therefore, a separate Annex provides information on the validation results of the ACTIVE and PASSIVE products, summarising the key findings from these activities.
  2. Annex B: The main body of the report provides some comparison between the current C3S version (v202312) and the previous version (v202212). Annex B provides a more complete analysis to demonstrate that the data have been produced correctly and show any differences between the products.
  3. Annex C: The main body of the report refers to the daily images provided through CDS. This section evaluates the aggregated product (monthly and 10-day /dekadal), which can also be downloaded.
  4. Annex D : Contains a brief discussion on the differences between satellite and reanalysis soil moisture products and their applications.

1.3.2. Quality Assurance for Soil Moisture

Some analysis presented in this report has been undertaken using software specifically designed for the C3S project. However, in parallel, an online validation service - Quality Assurance for Soil Moisture (QA4SM)6 - has been developed, which undertakes similar tasks and is used for most of the global validations.

The QA4SM service is an online validation service, which allows the traceable validation of state-of-the-art satellite-derived soil moisture products (C3S SM, SMAP, SMOS, H-SAF ASCAT SSM, ESA CCI SM, etc.) as well as any user uploaded data sets. The comparisons can be carried out against in situ and model/reanalysis reference data and other satellite products. CDR v202312 of C3S SM is available online for all users of the service (data files cannot be downloaded).

Validation results produced using the QA4SM service have a permalink assigned. This redirects users to archived, traceable, citable results on the Zenodo platform7. The relevant links are given in each chapter. The used QA4SM version was v2.7.0 (released 2024-05-16).

6  https://qa4sm.eu (resource validated 16th May 2024)

7 Multi-disciplinary open repository where datasets, documents, and other research materials can be located. https://zenodo.org/ (resource validated 16th May 2024)

2. Validation results

2.1. Spatial and temporal completeness

It is important to consider the spatial and temporal completeness and consistency of the product as these can be the key deciding factors for the users in terms of whether the product is suitable for their application.

The spatial and temporal coverage of the product is presented in terms of the number of available, valid daily observations. The highest coverage is expected in equatorial regions (due to the orbital paths of the satellites resulting in higher coverage), large parts of the southern hemisphere, India, and parts of the contiguous US and Europe. Lower data coverage is expected for regions affected by (seasonal) soil moisture freeze-thaw processes, mountainous areas, deserts, and areas affected by Radio Frequency Interference (RFI). Ice sheets and areas covered by rainforests are permanently masked out in all C3S SM products.

Figure 4 shows the data coverage of the v202312 COMBINED product, with the expected spatial (Fig. 4a) and temporal (Fig. 4b) patterns. Coverage increased over time, as more satellites suitable for measuring SM became available, and significantly improved with the addition of AMSR-E in 2002 and ASCAT in 2007. In some areas, almost full daily coverage has been achieved in recent years.

(a)

(b)

Figure 4: Data coverage of the C3S SM v202312 COMBINED product over land (excl. Antarctica) for the full period (1978-11 to 2023-12). 
Expressed as a percentage of the total number of days per period, (a) over the full period, (b) per month.


Figure 5 shows the same for the ACTIVE product. Compared to COMBINED, ACTIVE has an overall lower data coverage (fewer sensors), especially in the pre-ASCAT era (before 2007). Global gaps are found between March and July 2003, in which no ERS data is available. With the addition of ASCAT-B (the used NRT product is available after 2015) and ASCAT-C (2019), a significantly higher (temporal) coverage is achieved. ASCAT-A was decommissioned in November 2021. The increase in coverage in 2020/2021 (Fig. 5b) is likely related to changes in the orbit of ASCAT-A but does not affect the accuracy of the soil moisture retrieval.

Fig. 5a shows that additional permanent gaps are found in some regions (desert). This is in line with the underlying ESA CC SM v08.1 data and related to the seasonal triple collocation estimates. A more detailed discussion is given in Annex B .

(a)

(b)

Figure 5: Data coverage of the C3S SM v202312 ACTIVE product over land (excl. Antarctica) for the full period (1991-08 to 2023-12).
Expressed as a percentage of the total number of days per period, (a) over the full period, (b) per month.

Figure 6 shows the coverage of PASSIVE C3S SM. As expected, patterns are similar to the COMBINED product, but overall coverage is slightly lower. Compared to ACTIVE, no permanent gaps are found in deserts, although coverage is still low. 

(a)

(b)

Figure 6: Data coverage of the C3S SM v202312 PASSIVE product over land (excl. Antarctica) for the full period (1978-11 to 2023-12).
Expressed as a percentage of the total number of days per period, (a) over the full period, (b) per month.

2.2. Accuracy – Comparison against ISMN

2.2.1. Introduction

The C3S SM COMBINED dataset has been compared to the ISMN dataset and the correlations and ubRMSD between the two datasets are calculated. A mean and standard deviation matching is applied to bring satellite and in situ observations into the same value range. 

In addition to an overall comparison processed using the QA4SM service (Section 2.2.2 ), the comparison is also undertaken for different attributes of the soil moisture data (provided as metadata within the ISMN dataset). These are the sensor depth (Section 2.2.2.1 ), soil texture (Section 2.2.2.2 ), Köppen-Geiger climate classes (Section 2.2.2.3 ), and land cover (Section 2.2.2.4 ). For further information on the origin of these attributes, see Dorigo et al. (2011) .

The aim of assessing the accuracy for different attributes is to demonstrate the performance of the product under different conditions and demonstrate that KPIs DX.1 (where X is equal to 1 – 6) are met. See the PQAD [RD2], for the list of KPIs; repeated in this document in Table 4 .

The focus here is on the COMBINED product as this is the recommended product for most applications and designed to theoretically outperform the ACTIVE and PASSIVE products.

Validation runs from the following chapters are traceable8. The following validation runs are available on QA4SM and validation results are published on Zenodo:

8  All resources in the list were validated on 28 May 2024

2.2.2. Validation results

The comparison of C3S v202312 has been processed using the QA4SM service against ISMN FRMs at v20240314. The global map in Figure 7 shows the correlation (Pearson's) for each ISMN station to the nearest C3S grid cell; the same is shown for ubRMSD in Figure 8. These figures show the expected spatial patterns, with correlations between 0.6-0.7 and ubRMSD between 0.04-0.06 m3/m3 seen at most ISMN locations (as required below the minimum ubRMSD KPI threshold of 0.1 m3/m3). The performance COMBINED v202312 is equal to v202212 in this comparison.

(a)

(b)

Figure 7: Correlation (Pearson's) between C3S v202312 COMBINED and ISMN v20240314 FRMs for soil depths of 0–10 cm (a) and comparison of global scores with C3S v202212 COMBINED (b). Available via https://qa4sm.eu.
White boxes contain the computed validation scores for all ISMN sensors (N), coloured boxes indicate the lower and upper limits of the 95% confidence interval (CI) at the same locations.

(a)

(b)

Figure 8: ubRMSD between C3S v202312 COMBINED and ISMN v20240314 FRMs for soil depths of 0–10 cm (a) and comparison with C3S v202012 COMBINED (b). Available via https://qa4sm.eu.
White boxes contain the computed validation scores for all ISMN sensors (N), coloured boxes indicate the lower and upper limits of the 95% confidence interval at the same locations.


The same analysis is repeated for soil moisture anomalies from both data sets. Available observations in the period between 1990 and 2020 are averaged and subsequently smoothed with a 30-day moving window to create the climatological reference. Anomaly correlations indicate how well individual precipitation or drought events at the in situ stations are captured by the ~25 km satellite grid cells.

The ubRMSD is generally higher, and the correlation is lower respectively, compared to the absolute values (Figure 9). This is expected, as the well-represented seasonal signal component in both time series is no longer taken into account. However, as for the absolute values, the differences between the two C3S SM versions are negligible.


(a)

(b)

Figure 9: Correlation (a) and ubRMSD (b) between anomalies from C3S v202312 and v202212 COMBINED and ISMN v20240314 FRMs for soil depths of 0–10 cm. Available via https://qa4sm.eu.
White boxes contain the computed validation scores for all ISMN sensors (N), coloured boxes indicate the lower and upper limits of the 95% confidence interval at the same locations.

2.2.2.1. Stratification - Soil Depth

The correlation (Figure 10) and ubRMSD (Figure 11) between the last two versions of C3S SM COMBINED and the in situ datasets are presented for two different surface soil moisture depths (0-5 cm and 5-10 cm) using the absolute values (Fig. 10a, Fig. 11a) as well as anomalies (Fig. 10b, Fig. 11b).

The deeper sensors have a lower correlation and higher ubRMSD with C3S SM satellite observations than the sensors at shallower depths. This is expected as the product represents the first few centimeters of the soil surface (approx.). However, it may also be attributed to the decreasing errors associated with the ISMN data at these deeper depths - noting that the relationship seems to break at depths >1m (Gruber et al. 2013).

(a)

(b)

Figure 10: Correlation between ISMN FRMs and the two latest versions of C3S SM COMBINED product (v202312 and v202212) for soil depths of 0–5 cm and 5–10 cm for absolute values (a) and anomalies (b). 
Wide boxes contain the computed validation scores for all ISMN sensors (N), and narrow boxes indicate the lower and upper limits of the 95% confidence interval at the same locations.

(a)

(b)

Figure 11: ubRMSD between ISMN FRMs and the two latest versions of C3S SM COMBINED product (v202312 and v202212) for soil depths of 0–5 cm and 5–10 cm for absolute values (a) and anomalies (b).
Wide boxes contain the computed validation scores for all ISMN sensors (N), and narrow boxes indicate the lower and upper limits of the 95% confidence interval at the same locations.

2.2.2.2. Stratification - Soil Texture

The correlation and ubRMSD between the C3S dataset and the in situ datasets are presented in Figure 13 and Figure 14 for the different soil textures (fine, medium, and coarse; stratification provided from the ISMN dataset (Dorigo et al. 2011) and shown in Figure 12).


Figure 12: Soil texture classification used for the ISMN comparison. The percentage clay, silt, and sand are taken from the ISMN metadata which in turn is retrieved from the Harmonised World Soil Database (HWSD).

The product appears to perform best for medium and fine texture soils in terms of correlation and ubRMSD (for both depth layers). The spread in ubRMSD for fine soil textures is higher than for medium textures. Notice that the number of available sensors between soil texture classes changes significantly for different depths. While for most sensors in the top four centimeters "Coarse granularity" is assigned, "Medium granularity" is the predominant class for sensors below four centimeters. Compared to the previous version, correlation coefficients between satellite and in situ data remained mostly unchanged. In some cases, v202312 shows a slightly better agreement with ISMN than v202212.

 
Figure 13: Pearson's R between absolute values of C3S SM COMBINED and ISMN for different soil texture classifications and depths.

 
 
Figure 14: ubRMSD between absolute values of C3S SM COMBINED and ISMN for different soil texture classifications and depths.

The same overall level of agreement is found when comparing anomaly correlations and ubRMSD (Figure 15 and Figure 16). In some cases, v202212 slightly outperforms v202312.

 
Figure 15: Pearson's R between anomaly values of C3S SM COMBINED and ISMN for different soil texture classifications and depths.


 
Figure 16: ubRMSD between anomaly values of C3S SM COMBINED and ISMN for different soil texture classifications and depths.

2.2.2.3. Stratification - Köppen-Geiger climate classes

The correlation and ubRMSD between the C3S datasets and the in situ datasets are presented for different Köppen-Geiger classes (summarised as "Tropical", "Arid", "Temperate", "Continental" and "Polar") a global map of which is shown in Figure 17 (stratification provided from the ISMN metadata - see Dorigo et al. (2011)).

The correlation (absolute values in Figure 18, anomaly values in Figure 20) and ubRMSD (Figure 19 and Figure 21) between the in situ measurements and the C3S COMBINED product varies across different Köppen-Geiger classes. Comparing the performance for stations in "Arid" and "Continental" and "Temperate" climates (which are the three classes with a relatively large number of stations available), "Temperate" climate showing the highest correlation and second lowest ubRMSD with the satellite data in this comparison.

Notably "Tropical" and "Polar" perform surprisingly well in this comparison (compared to previous studies). This could either be due to the relatively low number of ISMN stations in these areas, but might also be related to the previously discussed selection of ISMN FRM sites (Chapter 1.2.1). Satellite products in these areas might perform better than previously assumed when taking into account the representativeness of in situ sites (land cover homogeneity, length of time series, etc.), and that the selection of representative reference data in these areas is more important than for temperate or arid zones to get a meaningful quality assessment.

The graphs show that in this case, the median ubRMSD values for all of the different climate classes are under the 0.1 m3/m3 minimum threshold set out in the KPIs; therefore the KPIs are achieved. The target GCOS threshold of 0.04 m3/m3 is approached and only reached in some cases.

Figure 17: Köppen-Geiger classes. The classes used in this assessment are summarised as "Tropical", "Arid" "Temperate", "Continental, "Polar". The figure is taken from http://hanschen.org/koppen .

 
 
Figure 18: Correlation (Pearson's) between absolute values of C3S SM COMBINED and ISMN for different climate classes in 0-10 cm depth.

 
 
Figure 19: ubRMSD between absolute values of C3S SM COMBINED and ISMN for different climate classes in 0-10 cm depth.

 

 
Figure 20: Correlation (Pearson's) between anomaly values of C3S SM COMBINED and ISMN for different climate classes in 0-10 cm depth.

 
 
Figure 21: ubRMSD between anomaly values of C3S SM COMBINED and ISMN for different climate classes in 0-10 cm depth.

2.2.2.4. Stratification - Landcover classes

The correlation and ubRMSD between the C3S dataset and the in situ datasets are aggregated for five different land cover classes: i) "Cropland", (ii) "Grassland", (iii) "TreeCover", (iv) "Urban Areas", (v) "Other"; stratification provided from the ISMN dataset (Dorigo et al. 2011).

Stations with land cover class "Grassland" assigned show the highest correlation (and lowest ubRMSD) with the satellite products among classes with a sufficient number of stations. Points with "TreeCover" on the other hand perform worst for both metrics between absolute values (Figure 22 and Figure 23) as well as anomalies (Figure 24 and Figure 25). The new C3S COMBINED product is on par with the previous version in all relevant cases.

The ubRMSD can be taken as a measure of accuracy and the KPIs specifically state the acceptable accuracy level of the product is between 0.01 and 0.05 m3/m3 for different land cover types. Therefore, overall the KPIs are met. The GCOS target of 0.04 is m3/m3 is approached and only reached in some cases.

 
Figure 22: Correlation (Pearson's) between absolute values of C3S SM COMBINED and ISMN for different land cover classes in 0-10 cm depth.

 
 
Figure 23: ubRMSD between absolute values of C3S SM COMBINED and ISMN for different land cover classes in 0-10 cm depth.

 

Figure 24: Correlation (Pearson's) between anomaly values of C3S SM COMBINED and ISMN for different land cover classes in 0-10 cm depth.

 
 
Figure 25: ubRMSD between anomaly values of C3S SM COMBINED and ISMN for different land cover classes in 0-10 cm depth.

2.2.3. Comparison to all previous versions 

A comparison of the correlation between satellite SM and in situ observations is provided in Figure 26 and Figure 27 for different versions of the C3S SM product. Absolute soil moisture values are used in this comparison. Only time stamps with valid observations in all data sets in the common period (1981-01 to 2019-12) are used. ERA5-Land temperature is used to remove all observations when the soil temperature is below 0° C. It can be seen that the overall correlation and ubRMSD between the in situ and satellite-derived soil moisture datasets improve with the release of new C3S SM product versions over time.

 
Figure 26: Correlation between the last four C3S SM versions, ERA5-Land, and ISMN FRMs.
White boxes contain the computed validation scores for all ISMN sensors (N), coloured boxes indicate the lower and upper limits of the 95% confidence interval at the same locations.


 
Figure 27: ubRMSD between the last four C3S SM versions, ERA5-Land, and ISMN FRMs.
White boxes contain the computed validation scores for all ISMN sensors (N), coloured boxes indicate the lower and upper limits of the 95% confidence interval at the same locations.


In addition, ISMN is compared to the latest four C3S SM versions and ERA5-Land, in 6 different sub-periods with a length of 4 years each i) 1996-1999, (ii) 2000-2003, (iii) 2004-2007, (iv) 2008-2011, (v) 2012-2015, (vi) 2016-2019. An increase in R of all C3S SM versions over time is found. Higher correlations with ISMN are found for v202212 than for previous versions. ERA5-Land shows overall better correspondence with the in situ data than the satellite products, especially in earlier periods. Relative differences between satellite and reanalysis products decrease significantly after 2015. This is likely due to the increased number of observations available as input to the C3S satellite products within this period. Especially the period after 2015 is positively affected by the addition of SMAP and shows higher correlations than the previous periods.

 
Figure 28: Correlation between ISMN FRMs (0-10 cm) and C3S satellite products and ERA5-Land for different periods.

In general, the products were deemed to agree well with in situ observations but were behind the performance of those obtained for Land Surface Model (LSM) simulation integrating observed precipitation, such as ERA5-Land. This may be due to the discrepancy between the installation depth of the in situ probes (typically 5 cm) and the typical depth of ~2 cm represented by the satellite-derived datasets (Dorigo et al. 2017).

2.3. Accuracy – Comparison against ERA5-Land reanalysis

C3S COMBINED v202312 has been compared against ERA5-Land top layer Soil Moisture (from 1981-01-01 to 2023-12-31, at 0:00 UTC). Metrics are computed from absolute soil moisture values, as well as anomaly values (reference period 1991-2020). Mean-standard deviation scaling is applied to remove biases between the satellite and reanalysis data prior to validation metrics computation.

Validation runs from this chapter are traceable. The following validation runs are available on QA4SM and published on Zenodo9:

9 All resources validated 24th May 2024

2.3.1. Comparison of absolute values

The absolute values C3S COMBINED v202312 and v202212 are compared against ERA5-Land top layer Soil Moisture. Correlation and ubRMSD are shown in Figure 29 and Figure 30 respectively.

Figure 29a shows expected spatial patterns in the correlation coefficient (Pearson's) between ERA5-Land and C3S SM v202312. There is a high positive correlation between the products in most temperate zones, with lower correlations being most prevalent in the northern, boreal regions. The deserts show little to no positive correlation. Comparison to the previous C3S SM version indicates a similar level of agreement between the reanalysis and satellite products. The same can be seen in terms of ubRMSD in Figure 30. ubRMSD is low (global median below 0.04 m3/m3), with a significant portion below the 0.1 m3/m3 threshold required in the KPIs (see Section 4).

(a)

(b)

Figure 29: Correlation (Pearson's) between absolute soil moisture values of the C3S SM v202312 COMBINED product and ERA5-Land top-level soil water content (a) and global comparison with the previous C3S SM version (b). 
White boxes contain the computed validation scores for all ERA5-Land grid cells (N), coloured boxes indicate the lower and upper limits of the 95% confidence interval.

(a)

(b)

Figure 30: ubRMSD between absolute soil moisture values of the C3S SM v202312 COMBINED product and ERA5-Land top-level soil water content (a) and global comparison with the previous C3S SM version (b).
White boxes contain the computed validation scores for all ISMN sensors (N), coloured boxes indicate the lower and upper limits of the 95% confidence interval at the same locations.

To compare the results for the current and previous versions spatially (v202312 vs v202212), differences in correlation and ubRMSD are computed. Improvements in correlation and reduction in ubRMSD of absolute values are shown in blue in Figure 31. In general, both versions show a very similar level of agreement with ERA5-Land, with maximum absolute difference rarely above 0.05 (R) and 0.005 (ubRMSD). The same comparisons for anomalies are shown in Figure 32.

(a)

(b)

Figure 31: Change in absolute SM values correlations (a) and ubRMSD (b) wrt. ERA5-Land top layer SM, between C3S SM COMBINED v202312 and v202212. Blue areas indicate higher agreement between the new satellite dataset version and reanalysis, red is the opposite.

(a)

(b)

Figure 32: Change in anomaly correlations (a) and ubRMSD (b) wrt. ERA5-Land top layer SM, between C3S SM COMBINED v202312 and v202212. Blue areas indicate higher agreement between the new satellite dataset version and reanalysis, red is the opposite.

2.4. Stability – Trend monitoring

Methods for monitoring the stability of the dataset are still under development. Here, preliminary results are presented to demonstrate the methods developed so far along with a discussion of how they will be developed in the future.

2.4.1. Accuracy evolution

To assess the evolution of the C3S SM dataset quality over time, a preliminary analysis is performed of the evolution of accuracy over the 2000 to 2023 period, using ISMN as reference measurements. In this case not only FRMs are used, but all available ISMN sensors between 0 and 5 cm depth. A bias correction is applied by scaling the satellite data to the in situ observations via mean and standard deviation matching.

Figure 33 shows the evolution of Pearson's R for different land cover types and climate conditions (described in Sections 2.2.2.3 and 2.2.2.4). It shows that the stability of C3S SM (COMBINED) varies, depending on the land cover type and climate class. An important factor to consider in this comparison is the number of ISMN stations available each year, which is represented by the small number below the boxes. Notably, there are more ISMN stations available over time. The product appears to be most stable for "Grasslands", while for other landcover classes, there is visible variation in the product, especially in earlier periods. In terms of climate classes, the highest stability is found for regions with temperate and arid climates ("Cf / Df", "BS").

Land cover classes

Climate classes

Figure 33: Accuracy evolution of C3S v202312 COMBINED between 2000 and 2023 in terms of Pearson's R based on left: aggregated land cover classes ((a) Cropland (b) Grassland (c) Tree Cover (d) Urban Areas) and right: aggregated climate classes ((a) Arid, (b) Temperate/Continental (no dry season), (c) Temperate/Continental (dry summer), (d) other). The numbers at the bottom indicate the number of ISMN stations used in the comparison.

Similar observations can be made in terms of ubRMSD (Figure 34). Here, note the expected widespread error for the "Urban areas" land cover. This is probably caused by the spatial resolution of C3S SM, where soil moisture networks close to densely populated areas are less representative of the whole C3S SM cell as satellite SM in these areas can be affected by landcover changes (city growth) and Radio Frequency Interference (RFI). The most unstable accuracy evolution is found for temperature/continental ("Csx/Dsx") and "other" climates (which include tropical and polar regions). These regions are strongly affected by seasonal freeze/thaw processes as well as dense vegetation, which can affect the retrieval quality when not sufficiently flagged. The low number of available in situ stations and their uneven global distribution should be considered.

The COMBINED product of C3S SM is below the ubRMSD KPI threshold of 0.1 m3/m3 in terms of median and Interquartile Range (IQR). Only in some cases, ubRMSD > 0.1 m³/m³ are found. For many stations located in "Grasslands" and arid climates, ubRMSD of 0.04 m³/m³ or less is achieved.  

Land cover classes

Climate classes

Figure 34: Accuracy evolution of C3S v202312 COMBINED between 2000 and 2023 in terms of ubRMSD based on left: aggregated land cover classes ((a) Cropland (b) Grassland (c) Tree Cover (d) Urban Areas) and right: aggregated climate classes ((a) Arid, (b) Temperate/Continental (no dry season), (c) Temperate/Continental (dry summer), (d) other). The numbers at the bottom indicate the number of ISMN stations used in the comparison.

The presented accuracy evolution results are the baseline for the estimation of stability trends in the dataset. Theil-Sen slopes were fitted to the median annual ubRMSD estimates. The so-found slope is supposedly representative of the long-term stability of the product. Note that this approach is currently under development and might change for future validation activities. Figure 35 shows the distribution of trends for different classes. At the tested locations for all land cover and climate classes, the change in accuracy metrics over time is very small, which indicates a stable SM product.

Land cover classes

Climate classes



Figure 35: left: Distribution of trends in ubRMSD in C3S SM v202312 COMBINED for different land cover classes: (a) Cropland (b) Grassland (c) Tree Cover (d) Urban Areas.
right: ubRMSd trends for different climate classes (right): (a) Arid, (b) Temperate/Continental (no dry season), (c) Temperate/Continental (dry summer), (d) other.
Tested against ISMN stations, where at least 3 years of accuracy evolution assessment was possible.

2.5. Time series analysis

Analysis of time series from a small number of locations provides insight into the behavior of the product for different climate and land cover types. Five points have been chosen for which ISMN in situ data are available (and were used in the above assessment). Details of the points are provided in Table 3 and they are shown on a global map in Figure 36.

Table 3: Details of locations chosen for time series analysis.

#

Ancillary

C3S data location

ISMN station location

Climate class

Land cover class

Country

GPI

Lat [°]

Lon [°]

Lat [°]

Lon [°]

1

Dsc

Sparse vegetation

USA

890047

64.625

-148.125

64.7232

-148.151

2

Cfa

Cropland

Australia

316669

-35.125

147.375

-35.1249

147.4974

3

BSk

Cropland

Spain

756697

41.375

-5.625

41.2747

-5.5919

4

Cfb

Grassland

Germany

810025

50.625

6.375

50.5149

6.3756

5

Cfa

Broadleaf forest

USA

733335

37.375

-86.125

37.2504

-86.2325

Note: all are classified as having ‘medium’ soil texture.

Figure 36: Locations of the points used in the time series comparison (points are given at the C3S GPI location).

The time series (temporally aggregated per month) for the individual locations for the ACTIVE, PASSIVE, and COMBINED products are given in Figure 37. In general, the time series appear to follow expected seasonal cycles at each location, i.e. winters are wetter and summers drier and, in the case of Grid point index (GPI) 890047 (which is located in Alaska), there are gaps in the data where the location is covered by snow each winter. For some locations (e.g. GPI 5 in Figure 36) a sudden drop in SM is visible, corresponding to the introduction date of SMOS, however, this is less pronounced in COMBINED and therefore most likely related to the scaling applied in generating the PASSIVE product. GPI 316669 shows a steep increase in ACTIVE over the last 3 years, which is most likely related to land cover changes and/or insufficient inter-calibration of ASCAT-C (compare Section A.3).

Figure 37: Time series of soil moisture (COMBINED and PASSIVE in [m3/m3], ACTIVE in [% sat.])  comparison for the COMBINED, ACTIVE, and PASSIVE products of C3S v202312 for the GPIs and land cover types stated for each plot (locations in Figure 36).

2.6. Uncertainty analysis

The algorithm used to develop the C3S soil moisture product utilizes triple collocation analysis to estimate signal-to-noise ratios (SNR) for the combination of different soil moisture observations (Gruber et al. 2017). Based on SNR estimates, weights are assigned to each sensor for merging. Since v202312, uncertainties are computed on a day-of-year (DOY) basis, meaning that different levels of intra-annual (seasonal) uncertainties are used to weight and subsequently merge sensors. 

Figure 38 shows time-latitude diagrams of all sensors merged in the v202312 COMBINED product.

 Figure 38: Intra-annual Signal-to-Noise Ratio estimates (gapfilled) derived from Triple Collocation Analysis for all sensors (day- and night-time overpass) merged in C3S SM COMBINED. The SNR is used to weight sensors for merging. Notice that for passive sensors used to generate the ICDR products (AMSR2, SMOS, SMAP, GPM) only night-time overpasses are used.


In combination with error propagation techniques, a per-pixel uncertainty is provided within the C3S SM product in the "sm_uncertainty" field (Figure 39). Since v202312, the 'sm_uncertainty' field therefore also shows seasonal variations in uncertainties. It is expected that the overall levels of uncertainty associated with the product reduce over time as additional, more accurate sensors are available (compare also Section 2.4.1). A higher level of agreement generally is found between sensors in the later periods. Throughout the product period, the uncertainties are always higher at latitudes where there is higher vegetation cover, for example at 10 degrees south. This is expected as soil moisture is harder to retrieve in these areas, and there is higher variance in the product where this is the case.

 
Figure 39: Monthly averages of the uncertainty variable associated with the C3S SM v202312 COMBINED product per latitude over time.

2.7. Break detection analysis

Break detection and correction methods are applied as part of the COMBINED product processing chain for the first time in C3S SM v202312. These methods apply statistical tests (Wilcoxon, 1945; Mann et al., 1947; Fligner et al., 1976) between temporal subsets of the observation time series to detect temporal, artificially caused, inhomogeneities ("breaks"). Breaks are defined as statistically significant differences in either mean or variance (or both) between (adjacent) sub-periods of an observation series. To differentiate between (artificial) breaks and natural changes, a reference dataset is required. ERA5 top level soil moisture is used as reference, and each grid cell time series of C3S SM is tested with respect to the nearest ERA5 time series. Break detection methods are described in more detail by Su et al. (2016) and Preimesberger et al. (2021). 

After a break is detected, a correction algorithm is applied. This algorithm modifies the C3S SM data so that the distribution functions of differences - between C3S SM and ERA5 - in the period before and after a detected break afterwards match. Different break correction algorithms, including the here used quantile matching approach, are described in (Preimesberger et al., 2021). Eight "sensor transition dates" (compare Figure 1) are tested in this process step. Detected breaks before and after the correction algorithm is applied are shown in Figure 40. About 80% of detected inhomogeneities are removed in the process, which is within the expected range.

Before Break Correction

After Break Correction

Figure 40: Detected inhomogeneities in mean (referred to as "WK" in the inset tables for "Wilcoxon test") and variance ("FK" for "Fligner-Killeen test") for 7 of 8 tested sensor transition dates (from top to bottom: 2015-03-31, 2012-10-01, 2010-01-15, 2007-10-01, 2002-06-19, 1998-01-01, 1991-08-05) in C3S SM v202312 COMBINED before (left column) and after (right column) the break removal algorithm is applied. Note: Results for 1987-07-09 are not shown due to low coverage. Rainforests are masked out in this plot (dark green).

3. Application(s) specific assessments

3.1. European State of the Climate Report

The C3S SM PASSIVE data are used in the "European State of the Climate 2023" report produced by ECMWF10. In the report, the C3S SM v202212 PASSIVE SM anomaly data (Figure 41b) are compared against ERA5-Land SM anomalies (Figure 41a). The anomalies from both data sets match well. Key events such as the drought over the Iberian Peninsula from February to May are captured by both data sets. Satellite retrievals during winter have a higher level of uncertainty due to potentially frozen soils, where soil moisture cannot be measured.

10 https://climate.copernicus.eu/esotc/2023 (resource validated 24th May 2024)

(a)

(b)

Figure 41: Monthly soil moisture anomalies for 2023 in ERA5-Land (a) and C3S v202212 PASSIVE (b). From the "Soil Moisture" monitoring chapter in "European State of the Climate in 2023"11.

11 The Soil Moisture chapter is available at https://climate.copernicus.eu/esotc/2023/soil-moisture 

4. Compliance with user requirements

The requirements for the C3S soil moisture product are a set of KPIs defined from consideration of user and GCOS requirements. The KPIs are shown in Table 4.

Table 4: Key Performance Indicators (KPIs) for the C3S Soil Moisture Product

KPI #

KPI Title

Performance Target and Unit of Measure

Accuracy KPIs

KPI.D1.1

CDR Radiometer with a daily resolution in latest quarter

Variable (0.01-0.10 m³ / m³), depending on land cover and climate

KPI.D2.1

CDR Scatterometer with a daily resolution in latest quarter

KPI.D3.1

CDR Combined with a daily resolution in latest quarter

KPI.D4.1

ICDR Radiometer with a daily resolution in latest quarter

KPI.D5.1

ICDR Scatterometer with a daily resolution in latest quarter

KPI.D6.1

ICDR Combined with a daily resolution in latest quarter

Stability KPIs

KPI.D1.2

CDR Radiometer with a daily resolution in latest quarter

0.01 m³ / m³ / y

KPI.D2.2

CDR Scatterometer with a daily resolution in latest quarter

KPI.D3.2

CDR Combined with a daily resolution in latest quarter

KPI.D4.2

ICDR Radiometer with a daily resolution in latest quarter

KPI.D5.2

ICDR Scatterometer with a daily resolution in latest quarter

KPI.D6.2

ICDR Combined with a daily resolution in latest quarter

An independent accuracy assessment has been conducted comparing the product against in situ observations with different conditions taken into account, i.e. soil depth, land cover etc. In general, there is variability in the correlation between the datasets, with correlations ranging from 0.4 to above 0.8, depending on the conditions and the locations of the in situ stations used. The ubRMSD, which can be directly taken as a measure of accuracy, is demonstrated to be below 0.10 m3 / m3 for all the different conditions analyzed, including land cover type (currently cropland, grassland, and tree cover are considered). Therefore, the minimum threshold KPI for the C3S product for accuracy has been met globally. However, the GCOS target requirement of 0.04 m3/m3 is only met under certain land cover (grassland) and climate (semi-arid) conditions. The median ubRMSD for most comparisons was found ~0.05 m3/m3. Notice, however, that the use of in situ data for satellite validation is complicated by the presence of representativeness errors, which inflates the actual errors. The effect of representativeness will be addressed in future quality assurance activities, as a finer categorization of the product skill according to land cover type or vegetation cover.

A comparison against the ERA5-Land reanalysis has also been undertaken to provide a wider global view of the product. The comparison against ERA5-Land has shown expected results, with the minimum KPI threshold of 0.1 m3/m3 being met in almost all regions (the exception being a few areas with high topographic complexity). The target of 0.04 m3/m3 is only reached in some areas. Similar to the comparison to in-situ observations, the median ubRMSD is found at around 0.05 m3/m3 in most comparisons.

The stability of the C3S product has been assessed in terms of the change in yearly accuracy compared to ISMN stations for all years after 2004. The accuracy between the products (ubRMSD) has been calculated per year and the trends in the accuracy were also analyzed. The minimum KPI threshold for stability of 0.05 m³/m³/y is met when assessed using this method for all tested stations. The target of 0.01 m³/m³/y is met in most cases, with vegetation (changes) leading to lower product stability in some locations. More methods to assess SM stability are currently under investigation and will be presented in future evaluation studies.

Annex A: Outcomes of the ACTIVE and PASSIVE quality control

A.1 Introduction

To demonstrate the differences between the active, passive, and combined products for the previous (v202212) and the current version (v202312), a summary of the comparisons of the dataset to the ISMN (FRM) data is shown in Table 5. For all validation runs, C3S SM has been scaled to ISMN using mean/standard deviation scaling. Only common observations/time stamps between both C3S SM versions are used in the comparison. Overall, performance has remained mostly unchanged between the latest two versions.

Table 5: Results of comparison against ISMN (0-10 cm) for the last two C3S SM dataset versions (median values).

Metric

ACTIVE
[% sat.]

PASSIVE
[m³/m³]

COMBINED
[m³/m³]


v202312

v202212

v202312

v202212

v202312

v202212

Correlation (Pearson's) [-]

0.589

0.589

0.668

0.663

0.6870.683

ubRMSD

0.062

0.062

0.056

0.057

0.03160.032


The ACTIVE and PASSIVE products of C3S SM v202312 were also compared to the reference datasets described in Section 2. As expected, the COMBINED product overall outperforms the PASSIVE and ACTIVE products as shown in Figure 42 for the comparison to ERA5-Land.

(a)

(b)

Figure 42: Intercomparison of the COMBINED, ACTIVE, PASSIVE product of C3S v202312, with ERA5-Land as the reference – plots shown are for Pearson's R (left) and ubRMSD (right). Only common locations and timestamps between all inter-compared products are considered.

Figure 43 and Figure 44 show differences in correlation and ubRMSD with respect to ERA5-LAND respectively, between the COMBINED and ACTIVE or PASSIVE products. The COMBINED product usually outperforms the ACTIVE and PASSIVE products. The ACTIVE product still performs well in (sub)tropical and temperate climates (with vegetation cover), while passive data is used in arid regions.

(a)

(b)

Figure 43: Difference in correlation with ERA5-LAND between the COMBINED and ACTIVE (a), resp. COMBINED and PASSIVE (b) products of C3S SM v202312.

(a)

(b)

Figure 44: Difference in ubRMSD with ERA5-LAND between the COMBINED and ACTIVE (a), resp. COMBINED and PASSIVE (b) products of C3S SM v202312.

A.2. Differences in long-term SM trends between the three products

Long-term trends in SM data sets are key to understanding the impact of climate change on global soil moisture. With more than 40 years of data, the assessment of long-term trends in C3S SM is possible. The COMBINED product has been used for analyses like this before (Dorigo et al. 2012). Ideally, one would expect similar trends in the ACTIVE, COMBINED, and PASSIVE products. However, as Figure 45 shows, there are differences and trends are often contradictory. Trends in ASCAT are known to be affected by long-term changes in land cover that the retrieval algorithm does not sufficiently account for and potentially affect the COMBINED product as well. A trend correction algorithm is currently being developed at EUMETSAT H-SAF.

The current suggestion is to use the COMBINED product for long-term studies. However, different trends indicate insufficient inter-calibration of some sensors or artificial trends within single products, as is demonstrated in Section A.3.


(a)

(b)

(c)

Figure 45: Long-term trends (Theil-Sen median slope)  from 1991-2020 in C3S SM v202312 COMBINED (a), ACTIVE (b) and PASSIVE (c). A Mann-Kendall significance test was applied and only statistically significant trends are shown here.


A.3. Wetting trends in the ASCAT data

An artificial wetting trend is found in ASCAT SSM. This affects the ACTIVE product (and the COMBINED product to a lesser extent) of C3S SM, as well as derived anomalies. Figure 46 shows anomalies over Europe for the year 2019 as in the ESA CCI SM v4 dataset. Note that this issue is still affecting ASCAT SSM and therefore C3S SM ACTIVE and to a lesser extent COMBINED at version v202312. Hence the issues described here still apply to the latest version. The right column shows the same anomalies when using an experimental version of ASCAT SSM with trend correction in the backscatter dry and wet reference in ESA CCI SM. Artificial trends in the radar observed backscatter signal are probably due to landcover changes and/or RFI (Ticconi et al. 2017). Cities, in particular, stand out in the SM anomaly maps of the uncorrected ACTIVE product. The issue was first detected in Europe but also affects other regions worldwide.

It is also noted that the issue seems most prominent in spring, and therefore may be (partly) related to the vegetation correction used in the ASCAT product.

The trend-corrected product of ASCAT SSM will be used in ESA CCI SM and C3S SM if officially released and provided by H-SAF.


Without ASCAT backscatter
trend correction

With ASCAT backscatter
trend correction

COMBINED

ACTIVE

PASSIVE

Figure 46: Change in ESA CCI SM v4 annual anomalies (for the year 2019) due to backscatter trend correction in ASCAT SSM. The climatological period for all plots is from 1991-2010. Note the different scales for each of the three products.

Annex B: Detailed comparison of C3S v202312 against C3S v202212

B.1 Introduction

This Annex provides a detailed comparison of the newest dataset version (v202312) against the previous version (v202212). The aim is to determine that the dataset has been made to specification. Differences between the products, which may be of interest to users when using the data in their applications, are highlighted.

B.2. Comparison of data coverage

Differences in data coverage between the current and previous versions are found (Figure 47, 48). All differences are within the expected range and due to changes to the ESA CCI SM merging algorithm that is used by C3S SM [RD8]. The largest decrease in available observations is found for the ACTIVE product (Fig. 47b, 48b) where for some grid cells, no data is available anymore in v202312. The reason for this is found in the newly introduced seasonal error estimation, where due to the low (soil moisture) signal strength in deserts, no (seasonal) TCA can be performed. Consequently, an increase in observations flagged as "low SNR" is found [RD8]. However, ACTIVE SM in deserts is considered unreliable due to sub-surface scattering effects (Wagner et al., 2023). Reduced coverage in these areas is therefore acceptable or beneficial to the overall data quality. 

To a lesser extent, the coverage is also reduced in the COMBINED product (Fig. 47a, 48a) for the same reasons. However, for some periods, an increase is found, which mainly originates from reduced overflagging in passive retrievals (Fig. 47c, 48c).


(a)

(b)

(c)

Figure 47: Change in (relative) number of valid observations between v202312 and v202212 for the COMBINED product.
Purple indicates an increase compared to the previous version, and orange a decrease in observations. Notice the different value range for ACTIVE.


(a)

(b)

(c)

Figure 48: Change in number of observations (daily) in C3S SM v202212 compared to v202012 in the period after 1991-01-01 for the COMBINED (a) ACTIVE (b) and PASSIVE (c) products.
Purple indicates an increase compared to the previous version, and orange a decrease in observations.

B.3. Comparison of time series

The locations, for which the observation time series are shown in Figure 49 for the different product versions, are the same as in Figure 36 . Overall, the products appear almost unchanged in most locations. For most points, the expected decrease in data coverage is found in ACTIVE.

(a)

(b)

(c)

Figure 49 : Time series of soil moisture for the different land cover classes considered (GPI locations shown in Figure 36). Showing the data for the COMBINED (a),  ACTIVE (b), and PASSIVE (c) products from v202312 and v202212 aggregated via 10-daily averages.

B.4. Comparison of soil moisture fields

Daily images for each of the ACTIVE, PASSIVE, and COMBINED products have been compared for C3S v202312 and v202212 (difference between them). Figure 50 shows that the largest differences are found in the ACTIVE product (corresponding to latitudes affected by reduced data coverage), some changes are found in COMBINED accordingly (partly due to the new break correction processing step), while PASSIVE remained similar to the previous version (Fig. 50c).


(a)

(b)

(c)

Figure 50: Absolute difference in soil moisture between C3S SM v202312 and v202212 for the daily COMBINED (a), ACTIVE (b) and PASSIVE (c) products.

B.5. Comparison of global statistics

To demonstrate the differences between the previous C3S version (v202212) and the current dataset (v202312), global statistics have been computed for each dataset version and are provided in Table 6. These are for the period after 2015-04-01.

As expected, the statistics remained mostly unchanged for all products between versions.

Table 6: Dataset statistics for the different C3S versions CDRs for the period from 2015-04-01 onwards for each product. The numbers given are the mean values across all points. Computed from daily data observations.

Metric

COMBINED [m3/m3]

ACTIVE [% sat.]

PASSIVE [m3/m3]

v202212

v202312

v202212

v202312

v202212

v202312

Mean

0.20

0.20

42.9

44.1

0.23

0.22

Median

0.21

0.21

42.2

43.99

0.20

0.19

Std. dev.

0.09

0.09

29.15

28.88

0.14

0.15

Max

1.0

1.0

100.0

100.0

1.0

1.0

Min

0.0

0.0

0.0

0.0

0.0

0.0

Annex C – Validation of Monthly and Dekadal Data

In addition to the daily C3S SM files, monthly and 10-day (dekadal) averaged files are provided. These include the number of averaged observations per file, the average SM (same unit as the daily data) and fields for the sensors and frequency bands that are merged.

Evaluation of these data was carried out with ERA5-Land as the reference (monthly and 10-day mean values from the original hourly data between 1981 and 2023). The absolute values were used in all cases. A mean/standard deviation scaling (to ERA5-Land) is applied to all satellite products in this validation bringing them into the same value range.

Correlations between monthly values (Figure 51 ) are higher than between daily ones. The COMBINED product also outperforms the ACTIVE and PASSIVE in this comparison, reaching a median R of 0.72 (median ubRMSD of 0.037 m3/m3 and therefore below the 0.04 m3/m3 KPI target). The same overall results are found for the dekadal data shown in Figure 52 .


(a)

(b)

Figure 51: Correlation (a) and ubRMSD (b) between monthly aggregated C3S SM products and ERA5-Land.

(a)

(b)

Figure 52: Correlation (a) and ubRMSD (b) between 10-day (dekadal) aggregated C3S SM products and ERA5-Land.


Notably, there is currently no threshold for a minimum number of observations per location in the daily images that are averaged. The representativeness of averaged images for the actual 10-day and monthly SM conditions depends on the data coverage for each aggregation period. This should be considered - especially for early periods of C3S SM and for areas with low data density - when using the aggregated files.

Annex D - Differences between Satellite and Reanalysis Soil Moisture products

C3S provides soil moisture datasets from different sources. ERA5 and ERA5-Land reanalysis, as well as the here-described satellite products, provide information on global surface soil moisture. For many users, the questions arises of what agreement level is expected between them and which dataset is best suited for a specific application.

While both data types describe surface soil moisture, there are fundamental differences in how they are produced. Both are viable information sources and generally, they agree well with each other (Chapter 2.2.3). Many studies, in which SM plays a crucial role, use both data sources to either confirm findings with a second, independent data source (e.g. ESotC 2023 report) or by using them in conjunction to build on the individual strengths of each product.

Satellite soil moisture is derived from measurements of electromagnetic waves emitted from Earth in the (water-sensitive) microwave domain. Retrieval models are applied to estimate water content in a scene from the raw physical measurements, which contain information from all contributing processes at the surface. Reanalyses use the interactions between atmospheric and terrestrial fluxes to predict soil moisture, relying on precipitation measurements for forcing. C3S satellite soil moisture and in situ soil moisture measurements are not assimilated in ERA5 or ERA5-Land. In situ soil moisture is not used in the production of C3S satellite soil moisture. It is therefore expected that reanalysis SM performs well in areas with an abundance of precipitation measurements to predict large-scale conditions that are close to the climatic mean. In contrast, satellite measurements can directly capture extreme events, and short-term changes, which models are more difficult to calibrate for.

As was shown in Chapter 2.2.3, a comparison to the most reliable reference data available (in situ SM) shows a slightly higher agreement with the reanalysis data, compared to the satellite data globally. However, as was shown in Chapter 2.2, the quality of satellite (and reanalysis data) varies locally. Comparison to ISMN data suggests, that while the reanalysis data performs overall slightly better on a global scale, for ~36% of ISMN sensors a higher agreement is found with the satellite data (Figure 53 ).

Figure 53: Correlation with ISMN in situ FRMs for C3S Satellite (y-axis) and ERA5-Land soil moisture (x-axis), data after 2015-04-01 are used in all cases. 
Points above the 1:1 line represent locations with a higher correlation between in situ and satellite, points below the line indicate a better performance of ERA5-Land.

There are several reasons why this can be the case. For example, irrigation signals are detected by satellites but are usually not accounted for by models (e.g., Zaussinger et al., 2019; Zappa et al., 2021). The same applies to other factors, especially in complex hydrological systems, such as soil moisture anomalies caused by lateral water influx from rivers. This was demonstrated by van der Schalie et al. (2022) for the Okavango Delta (Botswana), where some multi-year positive soil moisture anomalies were captured by the satellites but not adequately represented in ERA5-Land (Figure 54).

Figure 54: Anomalies for the Okavango River delta in satellite (top) and reanalysis (bottom) soil moisture. Figure taken from van der Schalie et al. (2022).


Reanalysis and satellite soil moisture are both used successfully for long-term trend studies (e.g. Vargas Zeppetello et al., 2024; Liu et al., 2023). However, when comparing long-term (multi-decadal) trends using simple (linear) methods, it is found that different satellite data sets (Figure 45) and model/reanalysis products contradict (Figure 55). It is therefore difficult to recommend one data type over the other for this application, which is the topic of ongoing research such as by Hirschi et al., 2023 (in review).

(a)

(b)

(c)

Figure 55 : 2000-2021 soil moisture trends in ERA5-Land (a), MERRA2 (b), and GLDAS Noah (c). Figure taken from [RD8].


Apart from the characteristics described above, there are other notable differences between satellite and reanalysis soil moisture (SM). C3S satellite SM includes uncertainty estimates, which can (and should) be used to assess the data quality and the uncertainty of therefore derived statistics. Another relevant point for many users concerns data gaps: Satellite observations often contain gaps, especially in winter, in locations where soil water content is frozen, whereas reanalyses provide gap-free fields. However, the quality of these estimates in winter is difficult to assess.

References

(JCGM), J. C. F. G. I. M. 2008. International vocabulary of metrology — Basic and general concepts and associated terms (VIM). VIM3: International Vocabulary of Metrology, 3, 104.

Dorigo, W., de Jeu, R., Chung, D., Parinussa, R., Liu, Y., Wagner, W., & Fernández-Prieto, D. (2012). Evaluating global trends (1988–2010) in harmonized multi-satellite surface soil moisture, 39

Dorigo, W., Himmelbauer, I., Aberer, D., Schremmer, L., Petrakovic, I., Zappa, L., Preimesberger, W., Xaver, A., Annor, F., Ardö, J., Baldocchi, D., Bitelli, M., Blöschl, G., Bogena, H., Brocca, L., Calvet, J.C., Camarero, J.J., Capello, G., Choi, M., Cosh, M.C., van de Giesen, N., Hajdu, I., Ikonen, J., Jensen, K.H., Kanniah, K.D., de Kat, I., Kirchengast, G., Kumar Rai, P., Kyrouac, J., Larson, K., Liu, S., Loew, A., Moghaddam, M., Martínez Fernández, J., Mattar Bader, C., Morbidelli, R., Musial, J.P., Osenga, E., Palecki, M.A., Pellarin, T., Petropoulos, G.P., Pfeil, I., Powers, J., Robock, A., Rüdiger, C., Rummel, U., Strobel, M., Su, Z., Sullivan, R., Tagesson, T., Varlagin, A., Vreugdenhil, M., Walker, J., Wen, J., Wenger, F., Wigneron, J.P., Woods, M., Yang, K., Zeng, Y., Zhang, X., Zreda, M., Dietrich, S., Gruber, A., van Oevelen, P., Wagner, W., Scipal, K., Drusch, M., & Sabia, R. (2021). The International Soil Moisture Network: serving Earth system science for over a decade. Hydrol. Earth Syst. Sci., 25, 5749-5804

Dorigo, W., Van Oevelen, P., Wagner, W., Drusch, M., Mecklenburg, S., Robock, A., & Jackson, T. (2011). A new international network for in situ soil moisture data. Eos, 92, 141-142

Dorigo, W., Wagner, W., Albergel, C., Albrecht, F., Balsamo, G., Brocca, L., Chung, D., Ertl, M., Forkel, M., Gruber, A., Haas, E., Hamer, P.D., Hirschi, M., Ikonen, J., de Jeu, R., Kidd, R., Lahoz, W., Liu, Y.Y., Miralles, D., Mistelbauer, T., Nicolai-Shaw, N., Parinussa, R., Pratola, C., Reimer, C., van der Schalie, R., Seneviratne, S.I., Smolander, T., & Lecomte, P. (2017). ESA CCI Soil Moisture for improved Earth system understanding: State-of-the art and future directions. Remote Sensing of Environment

Dorigo, W.A., Xaver, A., Vreugdenhil, M., Gruber, A., Hegyiová, A., Sanchis-Dufau, A.D., Zamojski, D., Cordes, C., Wagner, W., & Drusch, M. (2013). Global Automated Quality Control of In Situ Soil Moisture Data from the International Soil Moisture Network. Vadose Zone Journal, 12, 0

ENTEKHABI, D., REICHLE, R. H., KOSTER, R. D. & CROW, W. T. 2010. Performance Metrics for Soil Moisture Retrievals and Application Requirements. Journal of Hydrometeorology, 11, 832-840.

Fligner, M. A., & Killeen, T. J. (1976). Distribution-Free Two-Sample Tests for Scale. Journal of the American Statistical Association, 71(353), 210–213. https://doi.org/10.1080/01621459.1976.10481517

Gruber, A., Dorigo, W.A., Crow, W., Wagner, W., & Member, S. (2017). Triple Collocation-Based Merging of Satellite Soil Moisture Retrievals, 1-13

Gruber, A., Dorigo, W.A., Zwieback, S., Xaver, A., & Wagn, W. (2013). Characterizing Coarse-Scale Representativeness of in situ Soil Moisture Measurements from the International Soil Moisture Network. Vadose Zone Journal, 12

Gruber, A., Scanlon, T., van der Schalie, R., Wagner, W., and Dorigo, W.: Evolution of the ESA CCI Soil Moisture climate data records and their underlying merging methodology, Earth Syst. Sci. Data, 11, 717–739, https://doi.org/10.5194/essd-11-717-2019, 2019. 

Gruber, A., De Lannoy, G., Albergel, C., Al-Yaari, A., Brocca, L., Calvet, J. C., Colliander, A., Cosh, M., Crow, W., Dorigo, W., Draper, C., Hirschi, M., Kerr, Y., Konings, A., Lahoz, W., Mccoll, K., Montzka, C., Muñoz-Sabater, J., Peng, J., Reichle, R., Richaume, P., Rüdiger, C., Scanlon, T., Van Der Schalie, R., Wingeron, J. P. & Wagner, W. (2020). Validation practices for satellite soil moisture retrievals: What are (the) errors? Remote Sensing of Environment, 244, 111806.

Hersbach, H., Bell, B., Berrisford, P., Hirahara, S., Horányi, A., Muñoz-Sabater, J., Nicolas, J., Peubey, C., Radu, R., Schepers, D., Simmons, A., Soci, C., Abdalla, S., Abellan, X., Balsamo, G., Bechtold, P., Biavati, G., Bidlot, J., Bonavita, M., De Chiara, G., Dahlgren, P., Dee, D., Diamantakis, M., Dragani, R., Flemming, J., Forbes, R., Fuentes, M., Geer, A., Haimberger, L., Healy, S., Hogan, R.J., Hólm, E., Janisková, M., Keeley, S., Laloyaux, P., Lopez, P., Lupu, C., Radnoti, G., de Rosnay, P., Rozum, I., Vamborg, F., Villaume, S., & Thépaut, J.-N. (2020). The ERA5 global reanalysis. Quarterly Journal of the Royal Meteorological Society, 146, 1999-2049

Hirschi, M., Crezee, B., Stradiotti, P., Dorigo, W., and Seneviratne, S. I.: Characterising recent drought events in the context of dry-season trends using state-of-the-art reanalysis and remote-sensing soil moisture products, EGUsphere [preprint], https://doi.org/10.5194/egusphere-2023-2499, 2023.

Liu, Y.Y., Dorigo, W.A., Parinussa, R.M., De Jeu, R.A.M., Wagner, W., McCabe, M.F., Evans, J.P., & Van Dijk, A.I.J.M. (2012). Trend-preserving blending of passive and active microwave soil moisture retrievals. Remote Sensing of Environment, 123, 280-297

Liu, Y., Yang, Y., & Song, J. (2023). Variations in global soil moisture during the past decades: Climate or human causes? Water Resources Research, 59, e2023WR034915. https://doi.org/10.1029/2023WR034915

Mann, H. B., & Whitney, D.R., “On a test of whether one of two random variables is stochastically larger than the other”, The Annals of Mathematical Statistics, Vol. 18, pp. 50-60, 1947.

Muñoz-Sabater, J., Dutra, E., Agustí-Panareda, A., Albergel, C., Arduini, G., Balsamo, G., Boussetta, S., Choulga, M., Harrigan, S., Hersbach, H., Martens, B., Miralles, D.G., Piles, M., Rodríguez-Fernández, N.J., Zsoter, E., Buontempo, C., & Thépaut, J.N. (2021). ERA5-Land: a state-of-the-art global reanalysis dataset for land applications. Earth Syst. Sci. Data, 13, 4349-4383

Pasik, A., Gruber, A., Preimesberger, W., De Santis, D., & Dorigo, W. (2023). Uncertainty estimation for a new exponential filter-based long-term root-zone soil moisture dataset from C3S surface observations. EGUsphere, 2023, 1-32

PEEL, M. C., FINLAYSON, B. L. & MCMAHON, T. A. 2007. Updated world map of the Köppen-Geiger climate classification. Hydrol. Earth Syst. Sci., 11, 1633-1644.

Plummer, S., Lecomte, P., & Doherty, M. (2017). The ESA Climate Change Initiative (CCI): A European contribution to the generation of the Global Climate Observing System. Remote Sensing of Environment

Preimesberger, W., Scanlon, T., Su, C.-H., Gruber, A., & Dorigo, W. (2021). Homogenization of Structural Breaks in the Global ESA CCI Soil Moisture Multisatellite Climate Data Record. IEEE Transactions on Geoscience and Remote Sensing, 59(4), 2845-2862. https://doi.org/10.1109/TGRS.2020.3012896

Su, C.-H., Ryu, D., Dorigo, W., Zwieback, S., Gruber, A., Albergel, C., Reichle, R. H., and Wagner, W. (2016), Homogeneity of a global multisatellite soil moisture climate data record, Geophys. Res. Lett., 43, 11,245–11,252, doi:10.1002/2016GL070458.

Ticconi, F., Anderson, C., Figa-Saldana, J., Wilson, J.J.W., & Bauch, H. (2017). Analysis of Radio Frequency Interference in Metop ASCAT Backscatter Measurements. Ieee Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 10, 2360-2371

van der Schalie, R., van der Vliet, M., Albergel, C., Dorigo, W., Wolski, P., and de Jeu, R.: Characterizing natural variability in complex hydrological systems using passive microwave-based climate data records: a case study for the Okavango Delta, Hydrol. Earth Syst. Sci., 26, 3611–3627, https://doi.org/10.5194/hess-26-3611-2022, 2022.

Vargas Zeppetello, L.R., Trevino, A.M. & Huybers, P. Disentangling contributions to past and future trends in US surface soil moisture. Nat Water 2, 127–138 (2024). https://doi.org/10.1038/s44221-024-00193-x

Wagner, W., Dorigo, W., de Jeu, R., Fernandez, D., Benveniste, J., Haas, E., & Ertl, M. (2012). Fusion of Active and Passive Microwave Observations To Create an Essential Climate Variable Data Record on Soil Moisture. ISPRS Annals of Photogrammetry, Remote Sensing and Spatial Information Sciences, I-7, 315–321

Wagner, W., Lindorfer, R., Hahn, S., Kim, H., Vreugdenhil, M., Gruber, A., Fischer, M., & Trnka, M. (2023). Global Scale Mapping of Subsurface Scattering Signals Impacting ASCAT Soil Moisture Retrievals. TechRxiv. https://doi.org/10.34726/4782

Wilcoxon, F., Individual Comparisons by Ranking Methods, Biometrics Bulletin, Vol. 1, 1945, pp. 80-83. DOI:10.2307/3001968

Zaussinger, F., Dorigo, W., Gruber, A., Tarpanelli, A., Filippucci, P., and Brocca, L.: Estimating irrigation water use over the contiguous United States by combining satellite and reanalysis soil moisture data, Hydrol. Earth Syst. Sci., 23, 897–923, https://doi.org/10.5194/hess-23-897-2019, 2019. 

Zappa, L.; Schlaffer, S.; Bauer-Marschallinger, B.; Nendel, C.; Zimmerman, B.; Dorigo, W. Detection and Quantification of Irrigation Water Amounts at 500 m Using Sentinel-1 Surface Soil Moisture. Remote Sens. 2021, 13, 1727. https://doi.org/10.3390/rs13091727


This document has been produced in the context of the Copernicus Climate Change Service (C3S).

The activities leading to these results have been contracted by the European Centre for Medium-Range Weather Forecasts, operator of C3S on behalf of the European Union (Delegation Agreement signed on 11/11/2014 and Contribution Agreement signed on 22/07/2021). All information in this document is provided "as is" and no guarantee or warranty is given that the information is fit for any particular purpose.

The users thereof use the information at their sole risk and liability. For the avoidance of all doubt , the European Commission and the European Centre for Medium - Range Weather Forecasts have no liability in respect of this document, which is merely representing the author's view.

Related articles