Contributors: Richard Kidd (EODC GmbH), Christian Briese (EODC GmbH), Wouter Dorigo (TU Wien) Tracy Scanlon (TU Wien), Wolfgang Preimesberger (TU Wien), Robin van der Schalie (Vandersat)

Issued by: EODC GmbH/Richard A Kidd

Date: 14/06/2021

Ref: C3S_312b_Lot4_D1.S.1-2020_TRGAD_SM_i1.0

Official reference number service contract: 2018/C3S_312b_Lot4_EODC/SC2

Table of Contents

History of modifications

Issue

Date

Description of modification

Editor

i0.1

22/11/2020

Created from D1.S.1-2019.

RK

i1.0

14/06/2021

Finalised D1.S1-2020_TRGAD_LHC_i1.0.
Split to Soil Moisture TRGAD.

WP, RK

Related documents

Reference ID

Document

RD.1

Global Climate Observing System (2016) THE GLOBAL OBSERVING SYSTEM FOR CLIMATE: IMPLEMENTATION NEEDS, GCOS-200, https://library.wmo.int/doc_num.php?explnum_id=3417

RD.2

Merchant, C. J., Paul, F., Popp, T., Ablain, M., Bontemps, S., Defourny, P., Hollmann, R., Lavergne, T., Laeng, A., de Leeuw, G., Mittaz, J., Poulsen, C., Povey, A. C., Reuter, M., Sathyendranath, S., Sandven, S., Sofeiva, V. F. and Wagner, W. (2017) Uncertainty information in climate data records from Earth observation. Earth System Science Data, 9 (2). pp. 511-527. ISSN 1866-3516 doi: https://doi.org/10.5194/essd-9-511-2017

RD.3

Group for High Resolution Seas Surface Temperature Data Specification (GDS) v2, Casey and Donlon (eds.), 2012,

https://www.ghrsst.org/wp-content/uploads/2016/10/GDS20r5.pdf

RD.4

W. Dorigo, T. Scanlon, P. Buttinger, , A. Pasik, C. Paulik,R. Kidd, 2020. C3S D312b Lot 4, D3.SM.5-v2.0, Product User Guide and Specification (PUGS): Soil Moisture (v201912).

RD.5

R. van der Schalie, R. De Jeu, C. Paulik, W. Dorigo, T. Scanlon, A. Pasik, C. Reimer, R. Kidd, 2020. C3S D312b Lot 4 D1.SM.2-v2.0,.Algorithm Theoretical Basis Document (ATBD): Soil Moisture (v201912).

RD.6

W. Dorigo, T. Scanlon, W. Preimesberger, P. Buttinger, A. Pasik, R. Kidd, C. Chatzikyriakou, 2020. C3S D312b Lot 4 D2.SM.1_v2.0 Product Quality Assurance Document (PQAD): Soil Moisture.

RD.7

T. Scanlon, W. Dorigo , , W. Preimesberger, R. Kidd, C. Chatzikyriakou, 2020. C3S D312b Lot 4, D2.SM.2-v2.0, Product Quality Assessment Report (PQAR): Soil Moisture (v201912).

RD.8

D1.GL.2-v3.0 ATBD Area Change

RD.9

D1.GL.2-v3.0 ATBD Elevation and Mass Change

RD.10

ATBD CCI:
climate.esa.int/media/documents/glaciers_cci_ph2_d21_atbd_v26_161114.pdf

RD.11

ATBD CCI Soil Moisture: ATBD CCI: ESA Climate Change Initiative Plus, Soil Moisture, Algorithm Theoretical Baseline Document (ATBD), Supporting Product Version 06.1, D2.1 Version 2, 19-04-2021 See https://admin.climate.esa.int/media/documents/ESA_CCI_SM_RD_D2.1_v2_ATBD_v06.1_issue_1.1.pdf

RD.12

PUG CCI Soil Moisture: ESA Climate Change Initiative Plus, Soil Moisture Product User Guide (PUG), Supporting Product Version v06.1, Deliverable ID: D4.2 Version 2, 16-04-2021. See

https://admin.climate.esa.int/media/documents/ESA_CCI_SM_D4.2_v2_Product_Users_Guide_v06.1_i1.0.pdf

Acronyms

Acronym

Definition

AATSR

Advanced Along-Track Scanning Radiometer

AMI-WS

Active Microwave Instrument - Windscat (ERS-1 & 2)

AFRL

Air Force Research Laboratory

ALS DEM

Airborne Laser Scanner Digital Elevation Model

AMI

Active Microwave Instrument

AMSR2

Advanced Microwave Scanning Radiometer 2

AMSR-E

Advanced Microwave Scanning Radiometer-Earth Observing System

AntIS

Antarctic Ice Sheet

ASCAT

Advanced Scatterometer (MetOp)

ASTER

Advanced Spaceborne Thermal Emission and Reflection Radiometer

ASTER GDEM

ASTER Global Digital Elevation Model

ATBD

Algorithm Theoretical Baseline Document

ATSR-2

Along Track Scanning Radiometer 2

AVHRR

Advanced Very-High Resolution Radiometer

C3S

Copernicus Climate Change Service

CCI

Climate Change Initiative

CDF

Cumulative Distribution Function

CDM

Common Data Model

CDR

Climate Data Record

CDS

Climate Data Store

CF

Climate Forecast

CMA

China Meteorological Administration

CNES

Centre national d'études spatiales

DEM

Digital Elevation Model

DMSP

Defense Meteorological Satellite Program

DOD

Department of Defense

ECMWF

European Centre for Medium-Range Weather Forecasts

ECV

Essential Climate Variable

EODC

Earth Observation Data Centre for Water Resources Monitoring

ERS

European Remote Sensing Satellite (ESA)

ESA

European Space Agency

ESGF

Earth System Grid Federation

ESRI

Environmental Systems Research Insitute

ETM+

Enhanced Thematic Mapper plus

EUMETSAT

European Organisation for the Exploitation of Meteorological Satellites

FAO

Food and Agriculture Organization

FoG

Fluctuations of Glaciers

FPIR

Full-polarized Interferometric synthetic aperture microwave radiometer

FTP

File Transfer Protocol

GCOM

Global Change Observation Mission

GCOS

Global Climate Observing System

GDS

Glacier Distribution Service

GHRSST

Group for High Resolution Sea Surface Temperature

GIA

Glacial Isostatic Adjustment

GLCF

Global Land Cover Facility

GLIMS

Global Land Ice Measurements from Space

GLL

Grounding Line Location

GLS

Global Land Survey

GMB

Gravimetric Mass Balance

GMI

GPM Microwave Imager (GMI)

GPM

Global Precipitation Mission

GRACE

Gravity Recovery and Climate Experiment

GRACE-FO

Gravity Recovery and Climate Experiment Follow On

GrIS

Greenland Ice Sheet

GTN-G

Global Terrestrial Network for Glaciers

HDF

Hierarchical Data Format

H-SAF

Hydrological Satellite Application Facility (EUMETSAT)

HRV

High Resolution Visible

ICDR

Interim Climate Data Record

ICESat

Ice, Cloud and Elevation Satellite

IFREMER

Institut Français de recherche pour l'exploitation de la mer

IGOS

Integrated Global Observing Strategy

IMBIE

Ice sheet Mass Balance Intercomparison Exercise

InSAR

Interferometric SAR

IPCC

Intergovernmental Panel on Climate Change

ISRO

Indian Space Research Organisation

IV

Ice Velocity

JAXA

Dokuritsu-gyosei-hojin Uchu Koku Kenkyu Kaihatsu Kiko, (Japan Aerospace Exploration Agency)

KPI

Key Performance Indicators

L2

Retrieved environmental variables at the same resolution and location as the level 1 (EO) source.

L3

Level 3

LIDAR

Light Detection and Ranging

LOS

Line of Sight

LPDAAC

Land Processes Distributed Active Archive Center

LPRM

Land Parameter Retrieval Model

LSWT

Lake Surface Water Temperature

LWL

Lake Water Level

MERRA

Modern-Era Retrospective analysis for Research and Applications

MetOp

Meteorological Operational Satellite (EUMETSAT)

MetOp SG

Meteorological Operational Satellite - Second Generation

MSI

Multi Spectral Imager

MWRI

Micro-Wave Radiation Imager

NASA

National Aeronautics and Space Administration

NED

National Elevation Data

NetCDF

Network Common Data Format

NIR

Near Infrared

NISAR

NASA-ISRO SAR Mission

NOAA

National Oceanic and Atmospheric Administration

NRL

Naval Research Laboratory

NSIDC

National Snow and Ice Data Center

NWP

Numerical Weather Prediction

OE

Optimal Estimation

OLI

Operational Land Imager

PMI

Polarized Microwave radiometric Imager

PQAD

Product Quality Assurance Document

PQAR

Product Quality Assessment Report

PUG

Product User Guide

QA4ECV

Quality Assurance for Essential Climate Variables

RFI

Radio Frequency Interference

RGI

Randolph Glacier Inventory

RMSE

Root Mean Square Error

SAOCOM

SAtélite Argentino de Observación COn Microondas

SAF

Satellite Application Facilities

SAR

Synthetic Aperture Radar

SCA

Scatterometer

SEC

Surface Elevation Change

SLC

Single Look Complex

SLSTR

Sea and Land Surface Temperature Radiometer

SMAP

Soil Moisture Active and Passive mission

SMMR

Scanning Multichannel Microwave Radiometer

SMOS

Soil Moisture and Ocean Salinity (ESA)

SPIRIT

Stereoscopic survey of Polar Ice: Reference Images & Topographies

SPOT

Satellites Pour l'Observation de la Terre

SRTM

Shuttle Radar Topography Mission

SRTM DEM

SRTM Digital Elevation Model

SSM

Surface Soil Moisture

SSM/I

Special Sensor Microwave Imager

SST

Sea Surface Temperature

SWIR

Shortwave Infrared

TCA

Triple Collocation Analysis

TM

Thematic Mapper

TMI

TRMM Microwave Imager

TOPEX-Poseidon

Topography Experiment - Positioning, Ocean, Solid Earth, Ice Dynamics, Orbital Navigator

TOPS

Terrain Observation with Progressive Scan (S-1)

TRMM

Tropical Rainfall Measuring Mission

TU

Technische Universität

TU Wien

Vienna University of Technology

URD

User Requirements Document

USGS

United States Geological Survey

UTC

Universal Time Coordinate

VIIRS

Visible Infrared Imaging Radiometer Suite

VNIR

Visible and Near Infrared

VOD

Vegetation Optical Depth

WARP

Water Retrieval Package

WCOM

Water Cycle Observation Mission

WGI

World Glacier Inventory

WGMS

World Glacier Monitoring Service

WGS

World Geodetic System

WindSat

WindSat Radiometer

General definitions

Level 2 pre-processed (L2P): this is a designation of satellite data processing level. "Level 2" means geophysical variables derived from Level 1 source data on the same grid (typically the satellite swath projection). "Pre-processed" means ancillary data and metadata added following GHRSST Data Specification, adopted in the case of LSWT.

Level 3 /uncollated/collated/super-collated (L3U/L3C/L3S): this is a designation of satellite data processing level. "Level 3" indicates that the satellite data is a geophysical quantity (retrieval) that has been averaged where data are available to a regular grid in time and space. "Uncollated" means L2 data granules remapped to a regular latitude/longitude grid without combining observations from multiple source files. L3U files will typically be "sparse" corresponding to a single satellite orbit. "Collated" means observations from multiple images/orbits from a single instrument combined into a space-time grid. A typical L3C file may contain all the observations from a single instrument in a 24-hour period. "Super-collated" indicates that (for those periods where more than one satellite data stream delivering the geophysical quantity has been available) the data from more than one satellite have been gridded together into a single grid-cell estimate, where relevant.

Target requirement: ideal requirement which would result in a significant improvement for the target application.

Threshold requirement: minimum requirement to be met to ensure data are useful.

Scope of the document

This document aims to provide users with the relevant information on requirements and gaps for each of the given products within the Land Hydrology and Cryosphere service. The gaps in this context refer to data availability to enable the ECV products to be produced, or in terms of scientific research required to enable the current ECV products to be evolved to respond to the specified user requirements.

The ECV products addressed include the three Surface Soil Moisture products provided by the Soil Moisture Service; derived from merged active microwave satellites, derived from merge passive microwave satellites, and a product generated from merged active and passive microwave sensors.

Initially an overview of each product is provided, including the required input data and auxiliary products, a definition of the retrieval algorithms and processing algorithms versions; including, where relevant, a comment on the current methodology applied for uncertainty estimation. The target requirements for each product is then specified which generally reflect the GCOS ECV requirements. The result of a gap analysis is provided that identifies the envisaged data availability for the next 10-15 years, the requirement for the further development of the processing algorithms, and the opportunities to take full advantage of current, external, research activities. Finally, where possible, areas of required missing fundamental research are highlighted, and a comment on the impact of future instrument missions is provided.

Executive Summary

Soil Moisture
The C3S soil moisture product comprises a long-term data record called a CDR which runs from 1978 (PASSIVE and COMBINED) or 1991 (ACTIVE) to December 2019. This CDR is updated on a dekadal basis (approximately every 10 days) in an appended dataset called an ICDR. The CDR and ICDR products are provided as NetCDF 4 CF and each of the three products are generated with three temporal resolutions (daily, dekadal, monthly), meaning that the service provides a total of 18 soil moisture products.

The ACTIVE products rely on data from the Active Microwave Instrument (AMI) on ERS -1/2 and the Advanced SCATterometer (ASCAT) onboard the Meteorological Operational Satellites (MetOp-A, MetOp-B).

The PASSIVE products rely on microwave radiometers, and some 7 sensors are currently integrated in the product, with AMSR2 and SMOS based soil moisture retrievals forming the basis of the passive microwave near-real-time ICDR processing.

For the generation of the ACTIVE products the continuation beyond the current MetOp program will be provided by the approved MetOp Second Generation (MetOp-SG) program, which will start in 2021/22, and has a goal to provide observations until at least 2042.

Considering the PASSIVE products, although there are sufficient different sources of data, a continuation of L-band based soil moisture could become problematic due to possible data access restrictions for the Water Cycle Observation Mission (WCOM) and no approved follow-up for the Soil Moisture Active Passive mission (SMAP) or ESA's Soil Moisture and Ocean Salinity mission (SMOS) as of yet. Nevertheless, within ESA and Copernicus, continuation of L-band radiometer observations either as SMOS follow-up or Copernicus L-band mission, are being considered.

Whilst the current soil moisture products are already compliant with C3S target requirements (GCOS 2011 target requirements) and in many cases even go beyond, there is still a requirement to further develop the retrieval methodology based on user requirements including the needs of the community expressed in the European Space Agency (ESA) Climate Change Initiative (CCI). The future development covers algorithm improvements and satellite datasets that have already been evaluated, with many of these ongoing research activities and developments being undertaken within the ESA Climate Change Initiative (CCI) and CCI+ programmes.

In the long-term, some fundamental research is required in order to improve the soil moisture products even further. For the ACTIVE products some of these areas include intercalibration, estimation of diurnal variability, improved modelling of volume scattering, backscatter in arid regions, and respectively the impact of sub-surface scattering on soil moisture retrieval. For the PASSIVE products, activities include updated temperature from Ka-band observations, development of an independent, ancillary free, soil moisture dataset, and continuing research on error characterisation and stability assessment.

The Sentinel-1, SMAP, WCOM satellite missions and the two ESA Copernicus candidate missions (Microwave Radiometer Mission and an L-Band SAR Mission) are all expected to have substantial impact on the quality of soil moisture retrieval in the coming years. The inclusion of additional measurements from satellites missions such as GPM and the FengYun program are expected to additionally increase data coverage for the C3S Soil Moisture product.

Reliance on External Research
Since the C3S programme only supports the implementation, development and operation of the CDR processor, any scientific advances of the C3S products entirely rely on funding provided by external programmes, e.g. CCI+, H-SAF, Horizon2020. Thus, the implementation of new scientific improvements can only be implemented if external funding allows for it. This depends both on the availability of suitable programmes to support the R&D activities and the success of the C3S contractors in winning potential suitable calls.

1. Soil Moisture ECV Service

The C3S Soil Moisture production system provides the climate community with a stable source of soil moisture data derived from satellite observations through the Climate Data Store of the Copernicus Climate Change Service (C3S). The C3S soil moisture product comprises a long-term data record called a Thematic Climate Data Record. This CDR, product version v201912, is updated on a dekadal basis (approximately every 10 days) in an appended dataset called an Interim Climate Data Record (ICDR). Both the CDR and ICDR consist of three surface soil moisture datasets: ACTIVE, PASSIVE and COMBINED. The ACTIVE and the PASSIVE product are created by using scatterometer and radiometer soil moisture products, respectively; the COMBINED product is a blended product based on the former two datasets. The CDRs run from 1978 (PASSIVE and COMBINED) or 1991 (ACTIVE) to December 2019.

The target requirements are the same for each of the 18 products produced as part of the C3S soil moisture product. The requirements include the format of the data, the temporal and spatial resolution of the data, the accuracy and stability of the product, metadata requirements and other quality related requirements. The requirements may evolve throughout the product lifetime; in such a case, this document will be updated to reflect this evolution.

In this document an analysis is made that compares the current performance of the C3S Soil Moisture products against its potential in the future. This analysis is performed by (1) evaluating both the risk and opportunities of current and future satellite coverage and data availability, (2) the current fitness-for-purpose compared to the user requirements and how this will evolve in the upcoming years, and (3) ongoing and future research that would be beneficial for integration into the CDR and ICDR processing algorithms.

1.1. Introduction

This section provides the product specifications and target requirements for the C3S soil moisture product, which have been derived from community requirements as well as international standards. The purpose of this section is to provide these requirements independent of any assessments such that the requirements can be tracked as the product develops. As part of the cyclical process employed in the generation of the C3S product, the needs of the community and hence the requirements presented here will be updated as required.

1.2. Soil Moisture Products

1.2.1. Product descriptions

The C3S soil moisture product comprises a long-term data record called a CDR which runs from 1978 (PASSIVE and COMBINED) or 1991 (ACTIVE) to December 2018.  This CDR is updated on a dekadal basis (approximately every 10 days) in an appended dataset called an ICDR.  The theoretical algorithm and the processing implemented in the CDRs and ICDRs are exactly the same and the data provided is consistent between them.

Both the CDR and ICDR consist of three surface soil moisture datasets: The ACTIVE and the PASSIVE product are created by using scatterometer, and radiometer soil moisture products, respectively; the COMBINED product is a blended product based on the former two datasets.  The sensors used in the generation of the COMBINED product are shown in Figure 1. For each dataset the Daily, the Dekadal (10-days) mean, and the Monthly mean are available as NetCDF-4 classic format, using CF 1.6 conventions (Eaton et al.), and comprise global merged surface soil moisture images at a 0.25 degree spatial resolution. In total, there are 18 products available, as listed in Table 1.


Figure 1: Sensor time periods used in the generation of the C3S COMBINED soil moisture product.

The Daily files are created directly through the merging of microwave soil moisture data from multiple satellite instruments. The Dekadal and Monthly means are calculated from these Daily files. The Dekadal datasets feature a 10-day mean of a month, starting from the 1st to the 10th, from 11th to the 20th, and from 21st to the last day of a month, while the Monthly mean represents the soil moisture mean of all daily observations within each month.

A detailed description of the product generation is provided in the Algorithm Theoretical Basis Document (ATBD) [RD.5] with further information on the product given in the Product User Guide (PUG) [RD.4].The underlying algorithm is based on that used in the generation of the ESA CCI v04.4 product, which is described in relevant documents ((Dorigo et al., 2017), (Gruber et al., 2017), (Scanlon et al., 2019), (Liu et al., 2012)).  In addition, detailed provenance traceability information can be found in the metadata of the product.

Table 1: List of Soil Moisture Products

ACTIVE, PASSIVE or COMBINED?

CDR or ICDR

Temporal Resolution

ACTIVE

CDR

Daily

Dekadal

Monthly

ICDR

Daily

Dekadal

Monthly

PASSIVE

CDR

Daily

Dekadal

Monthly

ICDR

Daily

Dekadal

Monthly

COMBINED

CDR

Daily

Dekadal

Monthly

ICDR

Daily

Dekadal

Monthly

1.3. Soil Moisture Service: User Requirements

The target requirements are the same for each of the 18 products (listed in Table 1) produced as part of the C3S soil moisture product. The requirements are listed in Table 2.

Many of the requirements are derived from knowledge of the user community including the needs of the community expressed in the European Space Agency (ESA) Climate Change Initiative (CCI) User Requirements Document (URD) (Haas et al, 2018). Some of the requirements are derived from consideration of international standards and good practices, for example, the revisit time, product accuracy and product stability are those required by Global Climate Observing System (GCOS) (WMO, 2016).

The key users for the data are from the climate monitoring and modelling communities as well as policy implementation users. Such users were consulted as part of the CCI URD and hence user specific requirements are captured here.

Currently there are no threshold values assigned for the defined targets. Further work will consider the accuracy of the dataset required for different land cover classes and this work will consider the threshold targets for different cases. It is noted, however, that accuracy assessment using in situ data is complicated by the presence of representativeness errors, which inflate the differences between the measurements; these will need to be taken into account in setting such thresholds.

As part of the cyclical process employed in the generation of the C3S product, the needs of the community, and hence the requirements presented here, will be updated as required.

Table 2: Summary of C3S ECV Soil Moisture requirements showing target requirements

Requirement

Target

Product Specification

Variable of interest

Surface Soil Moisture

Unit

Volumetric (m³/m³)

Product aggregation

L2 single sensor and L3 merged products

Spatial resolution

50 km

Record length

>10 years

Revisit time

Daily

Product accuracy

0.04 – 0.1m³/m³ depending on land cover type

Product stability

0.01 m³/m³/y

Quality flags

Not specified

Uncertainty

Daily estimate, per pixel

Format Specification

Product spatial coverage

Global

Product update frequency

Monthly to annual

Product format

Daily images, Monthly mean images

Grid definition

0.25°

Projection or reference system

Projection: Geographic lat/lon
Reference system: WGS84

Data format

NetCDF, GRIB

Data distribution system

FTP, WMS, WCF, WFS, OpenDAP

Metadata standards

CF, obs4mips

Quality standards

QA4ECV

1.4. Soil Moisture Service: Gap Analysis

This section provides a Gap Analysis for the soil moisture product. The purpose of this section is to describe the opportunities, or obstacles, to the improvement in quality and fitness-for-purpose of the Soil Moisture CDR. In this section we address the data availability from existing space-based observing systems; development of processing algorithms; methods for estimating uncertainties; scientific research needs; and opportunities for exploiting the new generation of Sentinels.

1.4.1. Description of past, current and future satellite coverage

Figure 2 shows spatial-temporal coverage that is used for the construction of the CDR and ICDR for all three C3S Soil Moisture products (ACTIVE, PASSIVE, and COMBINED). An extensive description of these instruments and the data specifications can be found in the C3S ATBD [RD.5] (Chapter 1, Instruments). This gives an indication of the continuously changing availability of sensors over time as used in the production of the soil moisture data records. In the C3S ATBD [RD.5] (Chapter 3.3, Merging strategy) how this variability is taken into account and how this affects the quality of the final product is explained. The recent developments in the data availability for both scatterometers and passive radiometers are described in Section 1.4.1.1 and 1.4.1.2, and how this potentially affects the COMBINED product in 1.4.1.3.  

Figure 2: Spatial-temporal coverage of input products used to construct the CDR/ICDR (a) ACTIVE, (b) PASSIVE, (c) COMBINED. Blue colours indicate passive (radiometer), red colours active microwave sensors (scatterometers). The periods of unique sensor combinations are referred to as ‘blending period’. Modified from Dorigo et al. (2017).


1.4.1.1. Active

Active microwave observations used in the production of C3S soil moisture data products (see Table 3) are based on backscatter measurements from the European Remote Sensing Satellites (ERS) 1 and 2's Active Microwave Instrument (AMI) wind scatterometer, and the Advanced SCATterometer (ASCAT) onboard the Meteorological Operational Satellites (MetOp). The sensors operate at similar frequencies (5.3 GHz C-band) and share a similar design. ERS AMI has three antennas (fore- mid-, and aft-beam) only on one side of the instrument while ASCAT has them on both sides, which more than doubles the area covered per swath. ERS AMI data coverage is variable spatially and temporally because of conflicting operations with the synthetic aperture radar (SAR) mode of the instrument. In addition, due to the failure of the gyroscope of ERS-2, the distribution of scatterometer data was temporarily discontinued from January 2001 whereas in June 2003 its tape drive failed. Complete failure of ERS-1 and ERS-2 occurred in 2000 and 2011, respectively.

Two MetOp satellites (MetOp-A and MetOp-B) are currently flown in the same orbit, while MetOp-C was launched in 2018 to replace MetOp-A from 2022. From that time, MetOp-A will remain in orbit to serve as backup in case of failure of one of the other MetOp satellites. Continuation beyond the current MetOp program will be provided by the approved MetOp Second Generation (MetOp-SG) program, which will start in 2021/22 and has the goal to provide continuation of C-band scatterometer and other systematic observations for another 21 years, i.e., at least until 2042. Thus, no potential gap in data coverage from C-band scatterometer missions is foreseen for the next two decades. MetOp-C is not yet integrated in the MetOp-ASCAT CDR used as input to C3S.

Table 3: Current and envisaged active microwave instruments suitable for soil moisture retrievals

Satellite Sensor

Provider

Operation period

Used freq.

Extra information

ERS-1 AMI WS

ESA/IFREMER

1991 –2000

5.3 GHz

VV polarization; ERS AMI data coverage is variable spatially and temporally because of conflicting operations with the synthetic aperture radar (SAR) mode of the instrument. High resolution product (25x25 km) still under production by ESA

ERS-2 AMI WS

ESA/IFREMER

1997 - 2010

5.3 GHz

VV polarization; ERS AMI data coverage is variable spatially and temporally because of conflicting operations with the synthetic aperture radar (SAR) mode of the instrument. Due to the loss of gyroscopes in January 2001, data from 2001/01/17 to 2003/08/13 is lost; only reduced spatial coverage in sight of ground receiving stations after June 2003; Both nominal (50x50 km) and high resolution product (25x25 km) available.

MetOp-A/B/C ASCAT

EUMETSAT (Level 1B); HSAF (Level 2)

Since 2007 (MetOp-A) / Since 2012 (MetOp-B); Since 2018 (MetOp-C)

5.3 GHz

VV Polarization; Intercalibration between MetOp-B and MetOp-A NRT data is available only available after June 2015 because of which MetOp-B can only be used after this date. A backward processing of MetOp-B may be performed once intercalibrated data become available from H-SAF/EUMETSAT; In 2016, Metop-A has started to drift away from the 9:30 LST position

MetOp SG

EUMETSAT

2022-2042

5.3 GHz

Scatterometer (SCA) will have specifications very similar to those of ASCAT with additional cross-polarization (VH) measurements taken at 90° and 270° azimuth

L-Band SAR Mission

ESA

?

1.4 GHz

First steps are taken for a candidate Copernicus L-Band SAR Mission, which would be a follow up mission for SMOS.

1.4.1.2. Passive

Several passive microwave radiometers are available that can be used for the retrieval of soil moisture (Table 4), however due to differences in sensor specifications and data access not all are of interest for direct use within the soil moisture climate data record. In general, a lower frequency observation is preferred for soil moisture retrievals, e.g. C-band and L-band. For an in-depth overview of the impact of different frequencies on the quality of the soil moisture retrievals in the PASSIVE product, e.g. due to vegetation influences or radio frequency interference (RFI), see the C3S ATBD [RD.5] (Chapter 3.1.3, Known limitations).

Currently, AMSR2- and SMOS-based soil moisture retrievals form the basis of the passive microwave near-real-time ICDR processing. However, when these fail several other satellites are available for use. The most important of these sensors is SMAP (Entekhabi et al., 2010), the latest L-band mission, for which first test results show improved overall soil moisture retrievals for the PASSIVE product (Van der Schalie & De Jeu, 2016). This dataset is already included within the Climate Change Initiative - Soil Moisture framework and will be included in the next CDR version (v3.0). Other passive datasets that are foreseen to be included in future CDRs and ICDRs include GMI (X-Band) and FengYun-3B/D, although access restrictions for the latter products (as well as for additional WindSat records) could affect their inclusion into C3S soil moisture as additional historical and/or NRT products.

Table 4 also includes a list of future satellite missions and provides insight into the continuation of current satellite programs. Although there are enough different sources of data, a continuation of L-band based soil moisture could become problematic due to possible data access restrictions for WCOM (Shi et al., 2016) and no approved follow-up for SMAP (Entekhabi et al., 2010) or SMOS (Kerr et al., 2010) as of yet. Nevertheless, within ESA and Copernicus, continuation of L-band radiometer observations, either as SMOS follow-up or Copernicus L-band mission, are being considered.

Table 4: Historical, current and envisaged radiometers suitable for soil moisture retrievals


Satellite Sensor

Provider

Launch

Used freq.

Extra information

SSMI, SSMIS

NASA, DoD

Since 1991

18.7 GHz

Onboard satellites from the Defense Meteorological Satellite Program (DMSP), however with the latest satellite DMSP-F19 failing and only F16, F17 and F18 available but functioning past their expected life time, continuation is currently at risk. Also 18.7 GHz is not preferred for soil moisture retrievals.

WindSat

NRL, AFRL, DoD

2003

6.6, 10.7 GHz

Onboard the Coriolis satellite. Already active since 2003 and currently data access is restricted.

MWRI

CMA

Since 2008

10.7 GHz

Instrument carried on the FengYun-3 satellites. FY-3B/C/D (2010, 2013, 2017) are currently active. Follow up missions planned with end of life > 2028. Access to FengYun data is however restricted. Secondly, lower frequencies are preferred for soil moisture retrievals. FY-3B is included in the current version of ESA CCI SM and therefore planned for inclusion in future C3S SM CDRs. Due to data access restrictions it is expected that FY sensors are suitable for inclusion into the CDR, yet not the ICDR, as NRT data updates are currently not possible. The inclusion of Fy-3D is also foreseen.

SMOS MIRAS

ESA

2009

1.4 GHz

First L-band mission for soil moisture retrievals. Functioning properly but the design life was three years with a goal of five years. Part of the current climate data records.

AMSR2

JAXA

2012

6.9, 7.3, 10.7 GHz

Based on the AMSR-E sensor on the AQUA mission. AMSR2 is a sensor on the GCOM-W1 satellite. Still functioning properly, follow up is expected in 2019 with the launch of GCOM-W2. After that, GCOM-W3 is still uncertain and under discussion. Soil moisture derived from AMSR2 is part of the current CDR and ICDR.

GMI

NASA

2014

10.7 GHz

Part of the Global Precipitation Mission (GPM) satellite. Coverage only between 65°N and 65°S. Lower frequencies are preferred for soil moisture retrievals. This sensor is included in the current version of ESA CCI SM and therefore planned for inclusion into C3S SM.

SMAP

NASA

2015

1.4 GHz

Latest L-band mission specifically designed for soil moisture retrievals. Although the radar failed shortly after launch, the radiometer is functioning well. A SMAP based soil moisture product is integrated into CDR/ICDR v3. In the first instance, the lifetime expectancy of the mission was 3 years.

WCOM, FPIR and PMI

CAS

est. 2020

See extra info.

The payload of the Water Cycle Observation Mission (WCOM) satellite includes an L-S-C tri-frequency Full-polarized Interferometric synthetic aperture microwave radiometer (FPIR) and a Polarized Microwave radiometric Imager (PMI) covering 6.6 to 150 GHz. This wide range of simultaneous observations provide a unique tool for further research on soil moisture retrieval algorithms. The future accessibility of the data outside of China is however uncertain.

MWI

EUMETSAT

2022

18.7 GHz

Microwave Imager similar to SSMIS on board the MetOp-SG B satellites. 3 satellites expected to launch, first one in 2022.

Microwave Radiometer Mission

ESA

?

?

First steps are taken for a candidate Copernicus Imaging Microwave Radiometer Mission, which is expected to be a sensor similar to AMSR2.

1.4.1.3. Combined

Due to the wide range of available satellites (both active and passive) now and in the upcoming decade, and the flexibility of the system as explained by the merging strategy in the C3S ATBD [RD.5] (Chapter 3.3, Merging strategy), there is very little risk concerning the extension of the COMBINED product into the future. The current quality is not expected to reduce in the upcoming years, however , as recent research in the ESA CCI SM project showed, successful integration of SMAP soil moisture datasets will lead to further improvements in the COMBINED product after March 2015 (Gruber et al., 2019). 

1.4.2. Development of processing algorithms

This section is based on Chapter 1.4 in the PUGS [RD.4]. Table 5 provides the C3S Soil Moisture product target requirements adopted from the GCOS 2011 target requirements and shows to what extent these requirements are currently met by the latest C3S Soil Moisture products (v201912). As one can see, the CDR and ICDR products currently provided by the system are compliant with C3S target requirements and in many cases even go beyond. Further details on product accuracy and stability are provided in PQAD [RD.6] (methodology to assess) and PQAR [RD.7] (assessment).

Table 5: Summary of C3S Soil Moisture requirements, the specification of the current C3S products, and the target proposed by the consortium, Green shading indicates target requirement is obtained, Yellow shading indicates target requirement is being approached, Red shading indicates that target requirement is not achieved. Items highlighted in bold show where the target requirement has been exceeded

Requirement

C3S and GCOS target requirements

C3S Soil Moisture v201912Products

Product Specification

Parameter of interest

Surface Soil Moisture (SSM)

Volumetric Surface Soil Moisture

Unit

Volumetric (m³/m³)

Volumetric (m³/m³ (passive merged product, combined active +passive merged product); (% of saturation (active merged product)

Product aggregation

L2 single sensor and L3 merged products

Gridded L2 single sensor products (passive microwave products only); L3 merged active, merged passive, and combined active + passive products

Spatial resolution

50 km

25 km

Record length

>10 years

>40 years (1978/11 - running present)

Revisit time

Daily

Daily

Product accuracy

0.04 m³/m³

Variable (0.04-0.10 m³/m³), depending on land cover and climate (current assessment for various climates, land covers and texture classes based on in-situ data shows accuracy to be < 0.1 m³/m³)

Product stability

0.01 m³/m³/y

0.01 m³/m³/y (Assessment indicates stability to be within: to be formally assessed)

Quality flags

Not specified

Frozen soil, snow coverage, dense vegetation, retrieval failure, sensor used for each observation, overpass mode, overpass time, RFI

Uncertainty

Daily estimate, per pixel

Daily estimate, per pixel

Format Specification

Product spatial coverage

Global

Global

Product update frequency

Monthly to annual

10-daily (ICDR), and 12 monthly (CDR)

Product format

Daily images, Monthly mean images

Daily images, dekadal (10-day) mean, monthly mean images

Grid definition

0.25°

0.25°

Projection or reference system

Projection: Geographic lat/lon
Reference system: WGS84

Projection: Geographic lat/lon
Reference system: WGS84

Data format

NetCDF, GRIB

NetCDF 4

Data distribution system

FTP, WMS, WCF, WFS, OpenDAP

FTP/THREDDS

Metadata standards

CF, obs4mips

NetCDF Climate and Forecast (CF 1.7) Metadata Conventions; ISO 19115, obs4mips (distributed separately through ESGF)

Quality standards

QA4ECV

QA4ECV and QA4SM to be implemented

1.4.3. Methods for estimating uncertainties

The soil moisture uncertainty estimates are included in all C3S soil moisture products: ACTIVE, PASSIVE and COMBINED. A short overview is provided of how the uncertainties are estimated through the Triple Collocation Analysis (TCA, Gruber et al., 2016). Soil moisture uncertainty is the error standard deviation of the datasets estimated through TCA.

1.4.3.1. Triple Collocation Analysis

This section is based on CCI ATBD (Chapter 7.2.4), CCI PUG (Chapter 5.4.1) and Dorigo et al. (2017).
Triple collocation analysis is a statistical tool that allows the estimate of the individual random error variances of three datasets without assuming that any of them act as a supposedly accurate reference (Gruber et al. 2016a&b). This method requires the errors of the three datasets to be uncorrelated, therefore triplets always comprise of (i) an active dataset, (ii) a passive dataset, and (iii) the GLDAS-Noah land surface model, which are commonly assumed to fulfil this requirement (Dorigo et al., 2010). Error variance estimates are obtained as:


\[ \sigma^2_{\varepsilon_a} = \sigma^2_a - \frac{\sigma_{ap}\sigma_{am}}{\sigma_{pm}} \] \[ \sigma^2_{\varepsilon_p} = \sigma^2_p - \frac{\sigma_{pa}\sigma_{pm}}{\sigma_{am}} \]

where \( \sigma^2_{\varepsilon} \) denotes the error variance;  \( \sigma^2 \) and  \( \sigma \) denote the variances and covariances of the datasets; and the superscripts denote the active (a), the passive (p), and the modelled (m) datasets, respectively. For a detailed derivation see Gruber et al. (2016). Notice that these error estimates represent the average random error variance of the entire considered time period, which is commonly assumed to be stationary. Therefore, it only provides a single error estimate for a larger time period and not for each observation individually. In the ESA CCI SM production, TCA is applied to estimate the error variances of ACTIVE and PASSIVE. Unfortunately, TCA cannot be used to evaluate the random error characteristics of COMBINED, since, after blending ACTIVE and PASSIVE, an additional dataset with independent error structures would be required to complement the triplet. To address this issue, a classical error propagation scheme (e.g., Parinussa et al., 2011) is used to propagate the TCA-based error variance estimates of ACTIVE and PASSIVE through the blending scheme to yield an estimate for the random error variance of the final COMBINED product (Gruber et al., 2017):

\[ var(\varepsilon_c)=w_a^2var(\varepsilon_a)+w_p^2var(\varepsilon_p) \]

where the superscripts denote the COMBINED (c), ACTIVE (a) and PASSIVE (p) datasets, respectively; var(ε) denotes the error variances of the datasets; and w denotes the blending weights.

From the equation it can be seen that the error variance of the blended product is typically smaller than the error variances of both input products unless they are very far apart, in which case the blended error variance may become equal to, or only negligibly larger than, that of the better input product. However, the ACTIVE and PASSIVE input datasets of COMBINED are not perfectly collocated in time since the satellites do not provide measurements every day. In fact, there are days when either only ACTIVE or only PASSIVE provides a valid soil moisture estimate. In C3S, single-category observations are used to fill gaps in the blended product, but only if the error variance is below a certain threshold. Consequently, the random error variance of COMBINED on days with single-category observations is typically higher than that on days with blended multi-category observations. This results in an overall average random error variance of COMBINED that lies somewhere in between the random error variance of the single input datasets and the merged random error variance of all input products (estimated through error propagation) (Gruber et al., 2017).

Figure 3 shows global maps of the estimated random error variances of ACTIVE, PASSIVE, and COMBINED in the period where MetOp-A/B ASCAT, AMSR2, and SMOS are jointly available (July 2012-December 2015). The comparison with VOD from AMSR2 C-band observations (Figure 3d) shows that, at the global scale, error patterns largely coincide with vegetation density and that error variances are largely within thresholds defined by the C3S and GCOS user requirements (See Table 5). Even though the proposed solution to estimate random uncertainty seems to be accurate, it does not account for seasonally varying uncertainty, e.g. because of changes in vegetation. Therefore, a direct modelling of uncertainty within the production system would be favourable.


Figure 3: Average error variances of ESA CCI SM for ACTIVE (upper left), PASSIVE (upper right), and COMBINED (lower left) estimated through triple collocation and error propagation for the period July 2012-December 2015. Long-term (July 2012-December 2015) VOD climatology (lower right) from AMSR2 6.9 GHz observations (adapted from Dorigo et al., 2017).

1.4.4. Opportunities to improve quality and fitness-for-purpose of the CDRs

This section provides a brief overview of improvements that are being considered for introduction into the CDR and ICDR in a short term. This covers algorithm improvements and satellite datasets that have already been evaluated. Many of these ongoing research activities and developments are being undertaken within the ESA Climate Change Initiative (CCI) and CCI+ programmes, the continuation of which has not yet been officially approved. Given the large algorithmic dependency on the CCI programme, many of the following sections are based on the CCI ATBD (Scanlon et al., 2019).

Since the C3S programme only supports the implementation, development, and operation of the CDR processor, any scientific advances of the C3S products entirely rely on funding provided by external programmes, e.g. CCI+, H-SAF, Horizon2020. Thus, the implementation of new scientific improvements can only be implemented if external funding allows for it. The latter depends both on the availability of suitable programmes to support the R&D activities and the success of the C3S contractors in winning potential suitable calls.

1.4.4.1. ACTIVE products

The following issues are currently addressed by ASCAT soil moisture providers (H-SAF) and will improve the quality of C3S soil moisture when included in the active NRT input data streams.

1.4.4.1.1. Higher resolution sampling of ERS-1

An ERS-1 product with an improved spatial sampling (25x25 km) is provided by ESA and could be used to improve consistency between the derived ERS-1 soil moisture with ERS-2 and ASCAT soil moisture products. This is currently expected to be done by H-SAF within the CDOP-4 framework (2022-2027).

1.4.4.1.2. Intercalibration of MetOp-B and MetOp-A

Intercalibration between MetOp-B and MetOp-A NRT data is only available after June 2015, therefore MetOp-B can only be used after this date. A backward processing of MetOp-B may be performed once intercalibrated data becomes available from H-SAF/EUMETSAT.

1.4.4.1.3. Improved vegetation correction for ASCAT

An improved vegetation correction algorithm has been developed for ASCAT (Vreugdenhil et al., 2016) and is currently employed in the offline research product. The correction method has not yet been transferred to the NRT product distributed by HSAF. Once the new implementation is transferred to the operational NRT product, this will also be readily ingested into the CDR and ICDR.

1.4.4.1.4. Correction of artificial wetting trends

It is known that ASCAT soil moisture shows a positive (wetting) trend in some (densely populated) regions such as parts of Europe and Asia. The phenomenon is especially visible over cities and is probably related to RFI. Artificial trends in the data should be corrected while natural trends must be preserved. Multiple methods for this correction are currently being tested by H-SAF ASCAT SSM providers (Hahn et al. 2019).

1.4.4.1.5. Impact of sub-surface scattering on active soil moisture retrieval

It has long been noted that backscatter measurements over desert areas and semi-arid environments during a long dry spell exhibit an unusual behaviour that may lead to a situation where soil moisture from scatterometers is often less accurate than radiometer retrievals (Wagner et al., 2007, Gruhier et al., 2009, Hahn et al., 2020). Methods for the correct retrieval of soil moisture under the described conditions are currently being explored.

1.4.4.2. PASSIVE products
1.4.4.2.1. Introduction of SMAP soil moisture

As the SMAP observation frequency is similar to SMOS, the current algorithm as developed for SMOS (Van der Schalie et al., 2016 & 2017) can also be applied to the SMAP observations. SMAP SM is included in ESA CCI SM v5 (Pasik et al., 2020) and currently scheduled for inclusion in CDR v3. It is expected that the inclusion of SMAP will lead to increased observation density and soil moisture quality in C3S Soil Moisture.

1.4.4.2.2. Improved flagging of PASSIVE soil moisture under frozen soil conditions

A recently developed improved flagging strategy for radiometer products used in C3S soil moisture (van der Vliet et al. 2020) will improve SM retrieval especially for higher latitudes in future CDR versions. It allows consistent, satellite observation based assessment of the probability and subsequent masking of unreliable measurements due to the absence of liquid water in the soil under frozen conditions.

1.4.4.3. Merging
1.4.4.3.1. All products

1.4.4.3.1.1 Separate blending of climatologies and anomalies

Currently the merging scheme applies a relative weighting of datasets based on their relative error characteristics. However, studies have shown that different spectral components may be subject to different error magnitudes (Su et al., 2015). Therefore, investigations into the feasibility of blending the climatologies and the anomalies of the datasets separately are being undertaken.

1.4.4.3.1.1 Data density and availability

In CDR v3 gaps are only filled if the weight of the available product is above a relatively crudely defined empirical threshold. This threshold will be refined to find a best compromise between data density and product accuracy. In the current ESA CCI SM product (v6), SNR (merging weights) gaps are filled separately for different landcover types. Assuming that gap filling functions depend on the amount of vegetation present in a grid cell, fitting functions to fill missing pixels that fall under similar landcover types separately can improve gap filling results, particularly for regions with either no or very dense vegetation.

1.4.4.3.2. PASSIVE product

1.4.4.3.2.1 Using both night-time and day-time observations

Based on extensive product validation and triple collocation attempts to address the uncertainty of both ascending (daytime) and descending (night-time) modes will be made. Based on these results, this will guide decisions on how both observation modes can be considered in the generation of a single merged passive product, potentially leading to improved observation frequency with respect to the single descending mode used in the current PASSIVE product. An important step towards this approach was made by Parinussa et al. (2016).

1.4.4.3.2.2 Improved intercalibration of AMSRE and AMSR2 observations

The current constellation of sensors in C3S PASSIVE soil moisture omits an appropriate sensor to bridge the gap between AMSRE and ASMR2 observations. In ESA CCI SM v5 (Pasik et al., 2020) an improved intercalibration was achieved by matching (non-overlapping) subperiods of the two products. This resulted in the removal of a negative break in the merged passive data. The same improvement is planned for inclusion in CDR v3.0. Once additional passive products (FengYun 3B) are included, they can be used instead to bridge this gap.

1.4.4.3.3. ACTIVE product

1.4.4.3.3.1 Data gaps

In the framework of the C3S work, investigations into the potential use of ERS to fill gaps in the ASCAT time series will be undertaken.

1.4.5. Scientific Research needs

In the previous section, research activities that are already in an advanced stage of development and which could potentially be introduced into the CDR and ICDR in the short-term were discussed. However, in the long-term, some fundamental research is needed in order to improve the soil moisture products even further.

1.4.5.1. ACTIVE products
1.4.5.1.1. Inter-Calibration of Backscatter Data Records

To directly compare Level 2 surface soil moisture values retrieved from the ERS-1/2 AMI-WS and MetOp-A/B/C ASCAT, it is a pre-condition that these instruments have more or less exactly the same Level 1 calibration [RD.5]. Unfortunately, this is not yet the case owing to the fact that individual instrument generations underwent a somewhat different calibration procedure. Research is ongoing to improve the calibration between these sensors.

1.4.5.1.2. Estimation of Diurnal Variability

ASCAT measurements are performed for descending orbits (equator crossing 09:30, local time) and ascending orbits (equator crossing 21:30, local time). It has been noted that the backscatter measurements and, consequently, the Level 2 (L2) surface soil moisture retrievals from satellite platforms, although not dependent on temperature, show in some regions a difference between morning (i.e., day or sun-lit) and evening (i.e., night or dark) acquisitions (Friesen et al., 2012; Friesen et al., 2007). Currently, it is not clear if these observed diurnal differences are due to changes in the instrument between ascending or descending passes (e.g. due to the strong temperature differences in the sun-lit or dark orbital phases), shortcomings in the retrieval algorithm (e.g. neglecting diurnal differences in vegetation water content), or if these are just a natural expression of diurnal patterns of the surface soil moisture content. The underlying reasons for diurnal differences are to be investigated by comparing satellite ascending and descending orbit soil moisture retrievals.

1.4.5.1.3. Dry and Wet Crossover Angles

The crossover angle concept adopted in the retrieval method for scatterometers, states that at the dry and wet crossover angles, vegetation has no effect on backscatter (Wagner, 1998). These crossover angles have been determined empirically based on four study areas (Iberian Peninsula, Ukraine, Mali, and Canadian Prairies). Nevertheless, the empirically determined dry and wet crossover angles are used on a global scale in the surface soil moisture retrieval model. A known limitation of the global use of these crossover angles is that, depending on the vegetation type, or more precisely the evolution of biomass of a specific vegetation type, crossover angles may vary across the globe, which is not yet considered in the model. Furthermore, for some regions on the Earth's surface the crossover angle concept may not be applicable, in particular regions without vegetation cover (i.e., deserts). Recent investigations have shown that improved retrievals can be obtained by a local optimisation of cross-over angles (Pfeil et al., 2018).

1.4.5.1.4. Backscatter in Arid Regions

In arid regions, or more specifically in desert environments, it appears that the dry reference shows seasonal variations, which are assumed to reflect vegetation phenology. However, this cannot be true for desert environments, which are characterised by very limited or no vegetation at all. In principle, seasonal variations of the dry reference are desirable to account for backscatter changes induced by vegetation; referred to as vegetation correction. Vegetation correction is based upon changes in the slope parameter, which can be also observed in desert environments. These variations seem to have a big impact particularly in areas with very low backscatter. Hence, it needs to be clarified whether it is a real physical process, noise or something else reflected in the slope parameter.

1.4.5.2. PASSIVE products
1.4.5.2.1. Updated temperature input from Ka-band observations

Land surface temperature plays a unique role in solving the radiative transfer model and therefore directly influences the quality of the soil moisture retrievals. The current linear regression to link Ka-band measurements to the effective soil temperature has been re-evaluated by Parinussa et al. (2016) for daytime observations. An update to the linear regression for land surface temperature showed a significant increase in soil moisture retrieval skill. This research highlighted the importance and impact of correct temperature input into the algorithm. Further scientific work is needed to improve the surface temperature derived from microwave observations in order to significantly improve the skill of the soil moisture retrievals. Also, in order to remove model dependency for the L-band soil moisture retrievals that use modelled surface temperature as an input, investigations into combining the L-band observations with Ka-band observations from other satellites with similar overpass times are needed.

1.4.5.2.2. Development of a solely satellite based PASSIVE soil moisture data record

Within the climate community there is a strong preference for climate records that are solely satellite based. Any additional dataset that is used in a soil moisture retrieval algorithm could potentially lead to a dependency between a model and an observation. This is also why research was set up to investigate the possibility of developing an independent ancillary-free soil moisture dataset (Scanlon et al., 2019). Ancillary data could also have a strong impact on the spatial distribution of soil moisture. Artificial patterns of the 1 degree FAO soil property map are still visible in the original LPRM soil moisture product, however, these patterns disappear when only the dielectric constant is used. More research is needed to derive soil moisture from the dielectric constant records without making use of any ancillary datasets; with such an approach an independent dataset would be created that could be used as a benchmark for different modelled soil moisture datasets.

1.4.5.3. COMBINED products
1.4.5.3.1. Improved sensor inter-calibration

Currently, inter-calibration between active and passive datasets is done using CDF-matching against a long-term consistent land surface model. However, in order to achieve a full model independence of the CCI SM products alternative inter-calibration approaches will be investigated, for instance using lagged-variable based approaches or homogeneity tests (Su et al., 2015, 2016). Also the use of an L-band climatology as scaling reference for the COMBINED product is being investigated (Piles et al., 2018).

1.4.5.3.2. Improved homogeneity between sensor sub periods

The long-term consistency of the dataset can be improved through the operational implementation of break-detection and correction methods (Preimesberger et al., 2020). Inconsistencies in the soil moisture time series at sensor transitions in the COMBINED product can remain after the merging/scaling process and should be corrected in a separate post-processing step. However, the described methods rely on the use of additional (reanalysis) reference data for improved scaling.

1.4.5.4. Error characterisation
1.4.5.4.1. Estimation of random uncertainty per observation

The current C3S soil moisture product is generated with associated uncertainty estimates.  These estimates are based on the propagation of uncertainties, estimated with the triple collocation analysis, through the processing scheme; this process is described within the ATBD [RD.5]. Notice that these uncertainty estimates represent the average random error variance of the entire considered time period, which is commonly assumed to be stationary in the triple collocation. Future research shall focus on the estimation of the uncertainties of each individual measurement, which is driven, e.g., by the vegetation canopy density or the soil wetness conditions at the time of observation.

1.4.5.4.2. Stability assessment and correction

To test for inhomogeneities, the MERRA-2 data is compared to the C3S soil moisture; this procedure is described in the PQAD document (Dorigo et al., 2020). The inhomogeneity testing is achieved by first identifying potential locations of breakpoints in the time-series (for example where a change in sensors used occurs). Where the discontinuity values are greater than 1 % it is considered that this indicates a potential discontinuity in the time-series. The stability is then expressed in terms of the longest "stable" time-period within the dataset for each pixel. This gives a qualitative indication of the stability of the dataset, however, in future assessments of the dataset, the stability will be expressed in terms of m3 / m3 / y, thereby allowing demonstration against the KPIs. In addition, it is currently being investigated whether a break, once detected, can be corrected for. In this way, the "stable" time period can be extended.

1.4.6. Opportunities from exploiting the Sentinels and any other relevant satellite

As described in 1.4.1, there are many upcoming satellites relevant for soil moisture retrievals that are expected to be launched in the upcoming years. This section will give a more in-depth description of the instruments that could have a substantial impact on the quality of the soil moisture CDR and ICDR.

1.4.6.1. Sentinel-1

Soil Moisture retrieved through Sentinel-1 at 1km spatial resolution is currently in evolution at the Copernicus Global Land Service (Bauer-Marschallinger et al., 2019). Integration of a dataset like this could drastically improve the spatial resolution of the CDR, but only for data after 2014. So, for the data to be used, a strategy for handling a CDR with changing spatial resolution over time has to be developed. Sentinel-1 also has the potential to improve the soil moisture record spatial resolution using downscaling approaches together with other sensors. The combination with the ASCAT sensor seems promising (Bauer-Marschallinger et al., 2018) but, for integration into a CDR, the current approaches still need to overcome issues with temporal and spatial consistency.

1.4.6.2. Water Cycle Observation Mission (WCOM)

Although there are many uncertainties and concerns around the WCOM (Shi et al., 2016) mission, e.g. potential data accessibility issues, it would be a very interesting mission for the further development of the passive soil moisture retrieval algorithm. As described in Table 4, the payload of the WCOM satellite includes an L-S-C (1.4, 2.4 and 6.8 GHz) tri-frequency Full-polarized Interferometric synthetic aperture microwave radiometer (FPIR) and a Polarized Microwave radiometric Imager (PMI, 6 frequencies between 7.2 to 150 GHz). This wide range of simultaneous observations provide a unique tool for further research on soil moisture retrieval algorithms. Firstly, this allows for simultaneous retrieval of temperature from the Ka-band, which can be used in the soil moisture retrieval from the L-band observation, opposed to using modelled temperature. Secondly, this provides an opportunity for the first time to study S-band based soil moisture retrievals. Thirdly and most importantly, it provides a perfect tool for the development of a multi-frequency soil moisture retrieval approach based on L-, S-, C-, and X-bands, potentially leading to improved soil moisture retrievals.

1.4.6.3. Copernicus candidate missions under consideration

Two ESA missions that are currently under consideration as Copernicus candidate missions (http://missionadvice.esa.int/), a Microwave Radiometer Mission and an L-Band SAR Mission, would be an important step forward in safeguarding the future of the soil moisture climate records. With the upcoming MetOp-SG and Sentinel-1s, the active soil moisture retrievals have an expected satellite support up to 2040. However, for the passive soil moisture retrievals, and especially the development of long-term L-band based climate data records, the future is uncertain after SMOS and SMAP. For C-band frequencies and above, there is also some uncertainty after GCOM-W2. Therefore, these missions could form an important step in safeguarding the continuation of soil moisture climate data records with at least the same level of quality in the upcoming decades.

References

BAUER-MARSCHALLINGER, B., FREEMAN, V., CAO, S., PAULIK, C., SCHAUFLER, S., T. STACHL, MODANESI, S., MASSARI, C., CIABATTA, L., BROCCA, L. & WAGNER, W. 2019. Toward Global Soil Moisture Monitoring With Sentinel-1: Harnessing Assets and Overcoming Obstacles. IEEE Transactions on Geoscience and Remote Sensing, 57, 520-539.

BAUER-MARSCHALLINGER, B., PAULIK, C., HOCHSTÖGER, S., MISTELBAUER, T., MODANESI, S., CIABATTA, L., MASSARI, C., BROCCA, L. & WAGNER, W. 2018. Soil Moisture from Fusion of Scatterometer and SAR: Closing the Scale Gap with Temporal Filtering. Remote Sensing, , 10, 1030-1 - 1030-26.

CHUNG, D. 2017. C3S Product User Guide (PUG): Soil Moisture.

DORIGO, W., WAGNER, W., ALBERGEL, C., ALBRECHT, F., BALSAMO, G., BROCCA, L., CHUNG, D., ERTL, M., FORKEL, M., GRUBER, A., HAAS, E., HAMER, P. D., HIRSCHI, M., IKONEN, J., DE JEU, R., KIDD, R., LAHOZ, W., LIU, Y. Y., MIRALLES, D., MISTELBAUER, T., NICOLAI-SHAW, N., PARINUSSA, R., PRATOLA, C., REIMER, C., VAN DER SCHALIE, R., SENEVIRATNE, S. I., SMOLANDER, T. & LECOMTE, P. 2017. ESA CCI Soil Moisture for improved Earth system understanding: State-of-the art and future directions. Remote Sensing of Environment.

EATON, B., GREGORY, J., DRACH, B., TAYLOR, K. & HANKIN, S. NetCDF Climate and Forecast (CF) Metadata Conventions.

GRUBER, A., DORIGO, W., CROW, W. & WAGNER, W. 2017. Triple Collocation-Based Merging of Satellite Soil Moisture Retrievals. IEEE Transactions of Geoscience and Remote Sensing.

GRUHIER, C., DE ROSNAY, P., HASENAUER, S., HOLMES, T., DE JEU, R., KERR, Y., MOUGIN, E., NJOKU, E., TIMOUK, F., WAGNER, W. & ZRIBI, M. 2009. Soil moisture active and passive microwave products: intercomparison and evaluation over a Sahelian site. Hydrology and Earth System Sciences Discussions, 6, 5303-5339.

LIU, Y. Y., DORIGO, W. A., PARINUSSA, R. M., DE JEU, R. A. M., WAGNER, W., MCCABE, M. F., EVANS, J. P. & VAN DIJK, A. I. J. M. 2012. Trend-preserving blending of passive and active microwave soil moisture retrievals. Remote Sensing of Environment, 123, 280-297.

MITTELBACH, H., KIDD, R., DORIGO, W., DE JEU, R., WAGNER, W., SENEVIRATNE, S. I. & HAAS, E. 2013. ESA CCI Soil Moisture - User Requirements Document (URD).

Haas, E, Dorigo, W., Kidd, R., 2018. ESA CCI Soil Moisture - User Requirements Document (URD), version 1.4

PARINUSSA, R., DE JEU, R., VAN DER SCHALIE, R., CROW, W., LEI, F. & HOLMES, T. 2016. A Quasi-Global Approach to Improve Day-Time Satellite Surface Soil Moisture Anomalies through the Land Surface Temperature Input. Climate, 4, 50.

PFEIL, I., VREUGDENHIL, M., HAHN, S., WAGNER, W., STRAUSS, P. & BLÖSCHL, G. 2018. Improving the Seasonal Representation of ASCAT Soil Moisture and Vegetation Dynamics in a Temperate Climate Remote Sens., 10.

PILES, M., SCHALIE, R. V. D., GRUBER, A., MUÑOZ-MARÍ, J., CAMPS-VALLS, G., MATEO-SANCHIS, A., DORIGO, W. & JEU, R. D. Global Estimation of Soil Moisture Persistence with L and C-Band Microwave Sensors. IEEE International Geoscience and Remote Sensing Symposium (IGARSS 2018), 2018-07-22 - 2018-07-27 2018 Valencia, Spain. IEEE, 8259 - 8262.

PLUMMER, S., LECOMTE, P. & DOHERTY, M. 2017. The ESA Climate Change Initiative (CCI): A European contribution to the generation of the Global Climate Observing System. Remote Sensing of Environment.

VAN DER SCHALIE, R. 2017. C3S Algorithm Theoretical Basis Document: Soil Moisture.

WAGNER, W., DORIGO, W., DE JEU, R., FERNANDEZ, D., BENVENISTE, J., HAAS, E. & ERTL, M. 2012. Fusion of Active and Passive Microwave Observations To Create an Essential Climate Variable Data Record on Soil Moisture. ISPRS Annals of Photogrammetry, Remote Sensing and Spatial Information Sciences, I-7, 315--321.

WAGNER, W., NAEIMI, V., SCIPAL, K., DE JEU, R. & MARTINEZ-FERNANDEZ, J. 2007. Soil moisture from operational meteorological satellites. Hydrogeology Journal, 15, 121-131.

WMO 2016. The Global Observing System for Climate (GCOS): Implementation Needs.

De Nijs, A.H., Parinussa, R.M., De Jeu, R.A.M., Schellekens, J. and Holmes, T.R. (2015), “A methodology to determine radio-frequency interference in AMSR2 observations”, IEEE Transactions on Geoscience and Remote Sensing, vol. 53, no. 9, pp. 5148-5159.

Dorigo, W.A., Scipal, K., Parinussa, R.M., Liu, Y.Y., Wagner, W., De Jeu, R.A.M., and Naeimi, V. (2010), “Error characterisation of global active and passive microwave soil moisture datasets”, Hydrology and Earth System Sciences, vol. 14, pp. 2605 – 2616, doi: 10.5194/hess-14-2605-2010.

Dorigo, W., Wagner, W., Albergel, C., Albrecht, F., Balsamo, G., Brocca, L., Chung, D., Ertl, M., Forkel, M., Gruber, A., et al. (2017), “ESA CCI Soil Moisture for improved Earth system understanding: state-of-the art and future directions”, Remote Sensing of Environment, https://doi.org/10.1016/j.rse.2017.07.001

Entekhabi, D., Njoku, E.G., O’Neill, P.E., Kellogg, K.H., Crow, W.T., et al. (2010), “The Soil Moisture Active Passive (SMAP) Mission”, Proc. of the IEEE, vol. 98, no. 5, pp. 704-716, doi: 10.1109/JPROC.2010.2043918.

Friesen, J., Steele-Dunne, S.C., and Van de Giesen, N. (2012), “Diurnal differences in global ERS scatterometer backscatter observations of the land surface”, IEEE Transactions on Geoscience and Remote Sensing, vol. 50, no. 7, doi: 10.1109/TGRS.2012.2193889.

Friesen, J., Winsemius, H.C., Beck, R., Scipal, K., Wagner, W., and Van De Giesen, N. (2007), “Spatial and seasonal patterns of diurnal differences in ERS Scatterometer soil moisture data in the Volta Basin, West Africa”, IAHS Publication, vol. 316.

Gruber, A., Dorigo, W., Crow, W., and Wagner, W. (2017), “Triple Collocation-Based Merging of Satellite Soil Moisture Retrievals”, IEEE trans. geosc. rem. sens., DOI: 10.1109/TGRS.2017.2734070

Gruber, A., Su, C.H., Crow, W.T., Zwieback, S., Dorigo, W.A., & Wagner, W. (2016a). Estimating error cross-correlations in soil moisture data sets using extended collocation analysis. Journal of Geophysical Research: Atmospheres, 121(3), 1208-1219.

Gruber, A., Su, C.H., Zwieback, S., Crow, W.T., Wagner, W., & Dorigo, W. (2016b). Recent advances in (soil moisture) triple collocation analysis. International Journal of Applied Earth Observation and Geoinformation, 45, 200-211.

Gruber, A., Dorigo, W., van der Schalie, R., de Jeu, R., & Wagner, W. (in prep.). Improved blending of multi-satellite products into the ESA CCI soil moisture climate data record

Gruber, A., Scanlon, T., van der Schalie, R., Wagner, W., and Dorigo, W.: Evolution of the ESA CCI Soil Moisture climate data records and their underlying merging methodology, Earth Syst. Sci. Data, 11, 717–739, https://doi.org/10.5194/essd-11-717-2019, 2019.Gruhier, C., de Rosnay, P., Hasenauer, S., Holmes, T. R. H., de Jeu, R. A. M., Kerr, Y., ... Zribi, M. (2009). Soil moisture active and passive microwave products: intercomparison and evaluation over a Sahelian site. Hydrology and Earth System Sciences Discussions, 6, 5303-5339. DOI: 10.5194/hessd-6-5303-2009

Hahn, S., Vreugdenhil, M., Melzer, T., Forkel, M., Dorigo, W., & Wagner, W. (2019, January). Analyzing trends in backscatter and surface soil moisture observations from Metop ASCAT. In Geophysical Research Abstracts (Vol. 21).

Hahn, S., Wagner, W., Quast, R., & Salentinig, A. (2020, May). Subsurface scattering effects in the ASCAT soil moisture product. In EGU General Assembly Conference Abstracts (p. 20000).

Kerr, Y.H., Waldteufel, P., Wigneron, J.P., Delwart, S., Cabot, F., Boutin, J., Escorihuela, M.J., Font, J., Reul, N., Gruhier, C., Juglea, S.E., Drinkwater, M.R., Hahne, A., Martin-Neira, M., and Mecklenburg, S. (2010), “The SMOS mission: New tool for monitoring key elements of the global water cycle”, Proceedings of the IEEE, vol. 98, no. 5, doi: 10.1109/JPROC.2010.2043043.

Owe, M., de jeu, R., & Holmes, T. (2008). Multisensor historical climatology of satellite-derived global land surface moisture. Journal of Geophysical Research-Earth Surface, 113, F01002

Parinussa, R.M., De Jeu, R.A.M., Van der Schalie, R., Crow, W.T., Lei, F., and Holmes, T.R.H. (2016), “A Quasi-Global Approach to Improve Day-Time Satellite Surface Soil Moisture Anomalies through the Land Surface Temperature Input”, Climate, vol. 4, no. 50, doi:10.3390/cli4040050.

Parinussa, R.M., Meesters, A., Liu, Y.Y., Dorigo, W., Wagner, W., & de Jeu, R.A.M. (2011). Error Estimates for Near-Real-Time Satellite Soil Moisture as Derived From the Land Parameter Retrieval Model. IEEE Geoscience and Remote Sensing Letters, 8, 779-783

Pasik, A., Scanlon, T., Dorigo, W., de Jeu, R.A.M, Hahn, S., van der Schalie, R., Wagner, W., Kidd, R., Gruber, A. , Moesinger, L., Preimesberger, W. (2020): ESA Climate Change Initiative Plus – Soil Moisture Algorithm Theoretical Baseline Document (ATBD) Supporting Product Version v05.2

Preimesberger, W., Scanlon, T., Su, C., Gruber, A., and Dorigo, W. (2020) Homogenization of Structural Breaks in the Global ESA CCI Soil Moisture Multisatellite Climate Data Record," in IEEE Transactions on Geoscience and Remote Sensing, doi: 10.1109/TGRS.2020.3012896.

Scanlon, A. Pasik., W. Dorigo, R.A.M de Jeu, S. Hahn, R. van der Schalie, W. Wagner, R. Kidd, A. Gruber, L. Moesinger., W. Preimesberger (2019) CCI+ Soil Moisture Algorithm Theoritcal Basis Document (ATBD) v04.5, https://www.esa-soilmoisture-cci.org/sites/default/files/documents/public/CCI%20SM%20v04.5%20documentation/CCI2_Soil_Moisture_DL2.1_ATBD_v4.5.pdf

Shi et al. (2016), "The water cycle observation mission (WCOM): Overview," 2016 IEEE International Geoscience and Remote Sensing Symposium (IGARSS), Beijing, pp. 3430-3433. doi: 10.1109/IGARSS.2016.7729886Rastner, P., Bolch, T., Mölg, N., Machguth, H., Le Bris, R., Paul, F. (2012): The first complete inventory of the local glaciers and ice caps on Greenland. The Cryosphere, 6, 1483-1495. doi:10.5194/tc-6-1483-2012

Su, C.-H., and Ryu, D. (2015), “Multi-scale analysis of bias correction of soil moisture”, Hydrol. Earth Syst. Sci., vol. 17, doi:10.5194/hess-19-17-2015.

Su, C.-H., Zhang J., Gruber, A., Parinussa, R., Ryu, D., Crowd, W.T., and Wagner, W. (2016), “Error decomposition of nine passive and active microwave satellite soil moisture data sets over Australia”, Remote Sensing of Environment, vol. 182, doi: https://doi.org/10.1016/j.rse.2016.05.008.

Van der Schalie, R., and De Jeu, R.A.M. (2016), “SMAP Soil Moisture and Vegetation Optical Depth from the Land Parameter Retrieval Model”, as presented at the 3rd Satellite Soil Moisture Validation and Application Workshop, New York

Van der Schalie, R., De Jeu, R.A.M., Kerr, Y.H., Wigneron, J.-P., Rodriguez-Fernandez, N.J., Al-Yaari, A., Parinussa, R.M., Mecklenburg, S., and Drusch, M. (2017), “The merging of radiative transfer based surface soil moisture data from SMOS and AMSR-E”, Remote Sensing of Environment, vol. 189, doi: http://dx.doi.org/10.1016/j.rse.2016.11.026.

Van der Schalie, R., Kerr, Y.H., Wigneron, J.P., Rodriguez-Fernandez, N.J., Al-Yaari, and De Jeu, R.A.M. (2016), “Global SMOS Soil Moisture Retrievals from The Land Parameter Retrieval Model”, Int. J. Appl. Earth Observ. Geoinf., doi: http://dx.doi.org/10.1016/j.jag.2015.08.005.

Van der Schalie, R., Parinussa, R.M., De Jeu, R.A.M., Kerr, Y.H., Wigneron, J.P., Rodriguez-Fernandez, N.J., Al-Yaari, A., Mecklenburg, S., and Drusch, M. (2018), “Evaluation of three different data fusion approaches for the development of a soil moisture climate record from microwave satellite sensors”, Remote Sensing, ACCEPTED.

Vreugdenhil, M., Dorigo, W.A., Wagner, W., De Jeu, R.A.M., Hahn, S., and Van Marle, M.J.E. (2016), “Analyzing the Vegetation Parameterisation in the TU-Wien ASCAT Soil Moisture Retrieval”, IEE Transaction of Geoscience and Remote Sensing, vol. 54, no. 6., doi: 10.1109/TGRS.2016.2519842.

van der Vliet, M.; van der Schalie, R.; Rodriguez-Fernandez, N.; Colliander, A.; de Jeu, R.; Preimesberger, W.; Scanlon, T.; Dorigo, W. Reconciling Flagging Strategies for Multi-Sensor Satellite Soil Moisture Climate Data Records. Remote Sens. 2020, 12, 3439.

Wagner, W. (1998). Soil Moisture Retrieval from ERS Scatterometer Data. In, Institute for Photogrammetry and Remote Sensing. Vienna: Technical University of Vienna

Wagner, W., Blöschl, G., Pampaloni, P., Calvet, J.-C., Bizzarri, B., Wigneron, J.-P., Kerr, Y. (2007), “Operational readiness of microwave remote sensing of soil moisture for hydrologic applications”, Nordic Hydrology, vol. 38, no. 1.


This document has been produced in the context of the Copernicus Climate Change Service (C3S).

The activities leading to these results have been contracted by the European Centre for Medium-Range Weather Forecasts, operator of C3S on behalf of the European Union (Delegation Agreement signed on 11/11/2014 and Contribution Agreement signed on 22/07/2021). All information in this document is provided "as is" and no guarantee or warranty is given that the information is fit for any particular purpose.

The users thereof use the information at their sole risk and liability. For the avoidance of all doubt , the European Commission and the European Centre for Medium - Range Weather Forecasts have no liability in respect of this document, which is merely representing the author's view.

Related articles