Contributors: R. de Jeu (vandersat/planet labs), R. van der Schalie (vandersat/planet labs), C. Paulik (vandersat/planet labs), T. Frederikse (vandersat/planet labs), W. Dorigo (tuwien), T. Scanlon (tuwien), P. Stradiotti (tuwien), W. Preimesberger (tuwien), R. Kidd (EODC), C. Reimer (EODC), A. Dostalova (EODC) 

Issued by: EODC/Richard Kidd

Date: 17/01/2024

Ref: C3S2_312a_Lot4.WP2-FDDP-SM-v2_202312_SM_ATBD-v5_i1.1

Official reference number service contract: 2021/C3S2_312a_Lot4_EODC/SC1

<style>
.special_indent ul > li > ul {
    padding-left: 0;
}

.special_indent ul > li > ul > li > ul {
    padding-left: 1.5em;
}
</style>

History of modifications

List of datasets covered by this document

Related documents 

Acronyms 

General definitions 

Active (soil moisture) retrieval: the process of modelling soil moisture from radar (scatterometer and synthetic aperture radar) measurements. The measurand of active microwave remote sensing systems is called “backscatter”.

Backscatter is the measurand of “active” microwave remote sensing systems (radar). As the energy pulses emitted by the radar hit the surface, a scattering effect occurs and part of the energy is reflected back. The received energy is called “backscatter”, with rough surfaces producing stronger signals than smooth surfaces. It comprises reflections from the soil surface layer (“surface scatter”), vegetation (“volume scatter”) and interactions of the two. Under very dry soil conditions, structural features in deeper soil layers can act as volume scatterers (“subsurface scattering”).

Bias: “Bias is defined as an estimate of the systematic measurement error” GCOS-200 (WMO, 2016)

Brightness Temperature is the measurand of “passive“ microwave remote sensing system (radiometers). Brightness temperature (in degree Kelvin) is a function of kinetic temperature and emissivity. Wet soils have a higher emissivity than dry soils and therefore a higher brightness temperature. Passive soil moisture retrieval uses this difference between kinetic temperature and brightness temperature, to model the amount of water available in the soil of the observed area, while taking into account factors such as the water held by vegetation.

Level 2 pre-processed (L2P): this is a designation of satellite data processing level. “Level 2” means geophysical variables derived from Level 1 source data on the same grid (typically the satellite swath projection). “Pre-processed” means ancillary data and metadata added following GHRSST Data Specification.

Level 3 /uncollated/collated/super-collated (L3U/L3C/L3S): this is a designation of satellite data processing level. “Level 3” indicates that the satellite data is a geophysical quantity (retrieval) that has been averaged where data are available to a regular grid in time and space. “Uncollated” means L2 data granules have been remapped to a regular latitude/longitude grid without combining observations from multiple source files. L3U files will typically be “sparse” corresponding to a single satellite orbit. “Collated” means observations from multiple images/orbits from a single instrument combined into a space-time grid. A typical L3C file may contain all the observations from a single instrument in a 24-hour period. “Super-collated” indicates that (for those periods where more than one satellite data stream delivering the geophysical quantity has been available) the data from more than one satellite have been gridded together into a single grid-cell estimate, where relevant.

Passive (soil moisture) retrieval: the process of modelling soil moisture from radiometer measurements. The measurand of passive microwave remote sensing is called “brightness temperature”). The retrieval model in the context of Copernicus Climate Change Service (C3S) soil moisture is generally the Land Parameter Retrieval Model.

Polarisation: The polarization of microwaves refers to the orientation of the electric field vector of the transmitted beam with respect to the horizontal direction. the beam is said to be "V" polarized when the electric field vector oscillates along a direction perpendicular to the horizontal direction and "H" polarized when the electric field vector oscillates along a direction parallel to the horizontal direction.

Radiometer: Spaceborne radiometers are satellite-carried sensors that measure energy in the microwave domain emitted by the Earth. The amount of radiation emitted by an object in the microwave domain (~1-20 GHz). The observed quantity is called “brightness temperature” and depends on kinetic temperature of an object and its emissivity. Due to the high emissivity of water compared to dry matter, radiometer measurements of Earth’s surface contain information in the water content in the observed area.

Scatterometer: Spaceborne scatterometers are satellite-carried sensors that use microwave radars to measure the reflection or scattering effect produced by scanning a large area on the surface of the Earth. The initially submitted pulses of energy are reflected by the Earth’s surface depending on its geometrical and geophysical properties in the target area. The received energy is called “backscatter”. Soil moisture retrieval relies on the fact that wet soils have a higher reflectivity (and therefore backscatter) than dry soils due to the high dielectric constant of liquid water compared to dry matter.

Uncertainty: “Satellite soil moisture retrievals […] usually contain considerable systematic errors which, especially for model calibration and refinement, provide better insight when estimated separate from random errors. Therefore, we use the term bias to refer to systematic errors only and the term uncertainty to refer to random errors only, specifically to their standard deviation (or variance)” (Gruber et al., 2020)

Scope of the document

The Algorithm Theoretical Basis Document (ATBD) provides a detailed description of the algorithms that are used within the C3S Soil Moisture (SM) production system to produce the soil moisture Climate Data Record (CDR) and Interim Climate Data Records (ICDR). This document relates to C3S SM product versions v202312 for CDR and ICDR products.

Executive summary

The C3S SM production system is based on the algorithms initially developed within European Space Agency’s (ESA) Climate Change Initiative (CCI) Soil Moisture Project [D1] and has been updated and re-engineered to enable the provision of soil moisture retrievals on a near-real-time basis. In this context “near real time” refers to the provision of soil moisture information with a minimum delay of 10 days and a maximum of 23 days after sensing. This ATBD consists of a description of the input and auxiliary data used in the soil moisture retrievals, a description of the algorithms behind the soil moisture retrievals for both active and passive microwave observations, a description of the merging process and finally an overview of the output data.

More specifically this document relates to the C3S Soil Moisture production system used to generate product versions v202312 and is based on the algorithm developed for ESA CCI SM v08.1.

Chapter 1 provides an overview of the satellite instruments that are used for the production of the soil moisture data, which is a combination of both passive and active microwave sensors dating back to the year 1978. The satellites are selected based on the availability of compatible Level 3 products, respectively the applicability of the Land Parameter Retrieval Model (LPRM) in the case of microwave radiometers and their suitability for soil moisture retrieval in general (i.e. operating in preferably X, C or L-band frequencies).

Chapter 2 presents an overview of all the auxiliary data that is used during the generation and evaluation of the soil moisture record respectively. These datasets are partly used in the LPRM soil moisture retrieval for passive sensors, are partly used in the subsequent merging of the individual active and passive products as well as the evaluation of the merged products. The role of the in-situ data is to provide as good as possible reference observations to validate the final C3S SM products and rate their general performance. The validation process is described in more detail in the “Product Quality Assessment Document” (PQAD) [D4].

Chapter 3 gives an in-depth description of the algorithm used to retrieve soil moisture from passive microwave observations and points to relevant resources regarding active microwave retrievals from external operational sources. It contains a description of the main principles and methodologies in the LPRM, information on the parametrization for the most common frequency bands and the retrieval of soil moisture from daytime observations. An overview over currently known limitations of the model under certain conditions is given. The chapter also described the merging strategy applied in C3S soil moisture, i.e. how the retrievals from different satellite sensors are combined into a consistent merged soil moisture database, and how new data is repeatedly appended to this record. Chapter 4 briefly describes the output fields of the final soil moisture product.

Product Change Log

Table 1 provides an overview of the differences between different versions of the product up-to, and including, the current version v202312 (CDR v5.0). For a full list of all records provided in this version, see Table "List of datasets covered by this document".

Table 1: Changes in the product between versions.

Instruments

The CDR/ICDR each comprise of three products, being termed "active", "passive" and "combined", with each product being provided at three different temporal resolutions (daily, dekadal and monthly). A suite of data from various sensors are used in generating each product depending upon the temporal period and latitude of the retrieval. A schematic overview of the instruments used in the generation of the CDR/ICDR is provided in Figure 1, with information about the sensor characteristics being provided in Sections 1.1 and 1.2. 

Figure 1: Temporal coverage of input products used to construct the CDR/ICDR: ACTIVE (blue), PASSIVE (red), COMBINED (red and blue) products.

Passive Microwave Systems

Satellite missions from different space agency during the last 40+ years were equipped with microwave radiometers to measure electromagnetic radiation emitted by Earth at different frequencies in the microwave domain. A selection of frequency bands (L-, C-, X-, and Ku-band) allows the retrieval of soil moisture information from these measurements using the Land Parameter Retrieval Model (LPRM; described in more detail in Chapter 3.1). The model is applied to all hereafter listed passive sensors as part of the production of C3S SM. An overview over the data providers, data properties, used sensors and their main characteristics are given in the following chapters.

Nimbus-7 SMMR (NASA)

DMSP-SSM/I (NESDIS NOAA)

TRMM-TMI (NASA/JAXA)

AQUA-AMSR-E

Coriolis WindSat (Naval Research Laboratory)

GCOM-W1 AMSR-2 (JAXA)

SMOS (ESA)

SMAP (NASA)

FengYun 3B/C/D (CMA/NSMC)

GPM (NASA, JAXA)

Active Microwave Systems

Soil moisture retrieval from radar scatterometers was first done for ESA's ERS satellites (Wagner 1998; Wagner et al., 1999b). Since then, the Water Retrieval Package (WARP) was developed based on the TU-Wien method to derive soil moisture from backscatter measurements of Eumetsat's MetOp ASCAT sensors (H SAF, 2018), which is now distributed by H SAF as an operational product in near-real-time and used in the generation of C3S soil moisture. Soil moisture products from synthetic aperture radar measurements (such as Sentinel-1) are gaining popularity due to their higher spatial resolution and therefore potential new applications. They are, however, not yet part of the C3S records.

ERS AMI (ESA)

ASCAT Metop-A (EUMETSAT)

ASCAT Metop-B (EUMETSAT)

ASCAT Metop-C (EUMETSAT)

Input and auxiliary data

In the following an overview of the input and auxiliary data is provided. A division is made between data which is used as input for data production (Sections 2.1 and 2.2) and for validation activities (Section 2.3).

Advisory flags

The following section describes datasets that are used to generate the advisory flags provided as additional information to the soil moisture products. The advisory flags are not provided as part of the soil moisture datasets but only as datasets to support interpretation and analysis of the soil moisture products.

SSM/I-SSMIS EASE-Grid Daily Global Ice Concentration and Snow Extent

ERA-40

Global Lakes and Wetlands Database (GLWD)

Originating System	See Table 1 in Lehner and Döll (2004)
Data class	GIS database
Sensor Type and key technical characteristics	Organisation in three levels: Level 1 comprises the shoreline polygons of the 3067 largest lakes (surface area 50 km2) and 654 largest reservoirs (storage capacity 0.5 km3) worldwide and offers extensive attribute data. Level 2 contains the shoreline polygons of approx. 250,000 smaller lakes, reservoirs and rivers (surface area 0.1 km2), excluding all water bodies of level 1 Level 3 represents lakes, reservoirs, rivers, and different wetland types in the form of a global raster map at 30-second resolution, including all water bodies of levels 1 and 2.
Data Availability and Coverage	Global (except Antarctica)
Source Data Name and Product Technical Specifications	GLWD  https://www.worldwildlife.org/pages/global-lakes-and-wetlands-database
Data Quantity	~50 MB
Data Quality and Reliability	More information is provided in Lehner, B. & Döll, P., 2004.
Ordering and delivery mechanism	Download from FTP https://www.worldwildlife.org/pages/global-lakes-and-wetlands-database
Access conditions and pricing	GLWD is available for non-commercial scientific, conservation, and educational purposes.
Issues	N/A

GTOPO30

Merging Framework

The following data sets are used for merging L3 satellite soil moisture products into the harmonised C3S SM records.

Global Land Data Assimilation System (GLDAS) V2.0

GLDAS Noah is used in the Triple Collocation Analysis (TCA) step to estimate random uncertainties of the satellite products before merging. GLDAS Noah also acts as the scaling reference for the COMBINED product and is used there to harmonize all sensor time series via Cumulative Distribution Function (CDF) matching. 

Global Land Data Assimilation System (GLDAS) V2.1

GLDAS Noah is used in the TCA step to estimate random uncertainties of the satellite products before merging. GLDAS Noah also acts as the scaling reference for the COMBINED product and is used there to harmonize all sensor time series via CDF matching. 

ERA 5 / ERA5-Land

ECMWF ReAnalysis 5 (ERA5) and ERA5-Land variables are used as part of the soil moisture retrieval process (surface temperature) and act as the reference for break correction (surface soil moisture) in the production of the COMBINED product. 

Validation data

In situ data from the International Soil Moisture Network (ISMN)

ISMN in situ observations are the primary reference to assess the quality of the satellite based C3S soil moisture products. 

ERA5-Land

ERA5-Land (see Chapter 2.2.3) is the main globally available, gap-free reference source for evaluating C3S SM. While ERA5 is also used in the production of the data set (during break correction; Chapter 3.3.7), it was found that no features from ERA5 are introduced to the satellite data in that step. ERA5-Land is therefore considered to be independent and is therefore used as a reference for validation.

Algorithms

Passive Microwave Algorithm

This chapter is largely based upon the “Algorithm Theoretical Baseline Document (ATBD) D2.1 Supporting Version 08.1” [D1] document produced under ESA Climate Change Initiative Soil Moisture project.

Principles of the Land Parameter Retrieval Model

Brightness temperatures can be derived from passive microwave sensors with different radiometric characteristics. The observed brightness temperatures are converted to soil moisture values with the Land Parameter Retrieval Model (LPRM; van der Schalie et al., 2017). This model is based on a microwave radiative transfer model that links soil moisture to the observed brightness temperatures. A unique aspect of LPRM is the simultaneous retrieval of vegetation optical depth (VOD) in combination with soil moisture and surface temperature.

A result of this physical parameterisation is that any differences in frequency and incidence angle that exist among different satellite platforms are accounted for within the framework of the radiative transfer model based on global constant parameters (de Jeu et al., 2014). This important aspect makes LPRM suitable for the development of a long-term consistent soil moisture products which can be used in the ESA CCI SM products. 
The different processing steps of LPRM are described in detail in the next section, while Figure 2 presents a flowchart of the entire methodology.

Figure 2: Flowchart of the main processes of the Land Parameter Retrieval Model (LPRM). Soil moisture is solved when the observed brightness temperature equals the modelled brightness temperature as derived by the radiative transfer. 

Methodology

The thermal radiation in the microwave region is emitted by all natural surfaces and is a function of both the land surface and the atmosphere. According to LPRM the observed brightness temperature (T_b) as measured by a space borne radiometer can be described as:

$$T_{b,p} = \Gamma_a(T_{b\_s,p}+(1-e_{r,p})(T_{b\_d}+T_{b\_extra} \Gamma_a)\Gamma_v^2) + T_{b\_u} \quad Eqn. 1$$

Where Γ_a and Γ_v are the atmosphere and vegetation transmissivity respectively, T_{b_s} is the surface brightness temperature, e_r is the rough surface emissivity, T_{b_extra}, the extra-terrestrial brightness temperature and the T_{b_u} and T_{b_d} are the upwelling and downwelling atmospheric brightness temperatures. The subscript p denotes either horizontal (H) or vertical (V) polarization. The vegetation/atmosphere transmissivity is further defined in terms of the optical depth, τ_v/a, and satellite incidence angle, u, such that

$$\Gamma_{v/a} = exp(- \frac{\tau_{v/a}}{\cos u}) \quad Eqn. 2$$

The upwelling brightness temperature from the atmosphere is estimated as (Bevis et al. 1992):

$$T_{b\_u,p} = 70.2+0.72T_a(1-\Gamma_a) \quad Eqn. 3$$

Were T_a is the atmospheric temperature. In LPRM the downwelling Temperature (T_d) is assumed to be equal to the upwelling temperature (T_u) and the extraterrestrial temperature is set to 2.7 K (Ulaby et al. 1982). The radiation from a land surface (T_{b_s,p}) is described according to a simple radiative transfer (Mo et al. 1982):

$$T_{b\_s,p} = T_se_{r,p}\Gamma_v + (1-\omega)T_v(1-\Gamma_v)+ (1-e_{r,p})(1-\omega)T_v(1-\Gamma_v)\Gamma_v \quad Eqn. 4$$

Where T_s and T_v are the thermodynamic temperatures of the soil and the vegetation, ω is the single scattering albedo. LPRM uses the model of Wang and Choudhury (1981) to describe the rough surface emissivity as:

$$e_{r,p1} = 1-Q(r_{s,p2} + (1-Q)r_{s,p1})e^{-h \cos u}\quad Eqn. 5$$

Where $Q$ is the polarization mixing factor and $h$ the roughness height. $h$ is calculated using the related parameters $h_1$, $A_v$ and $B_v$, see Eqn. 6, to take into account the effects of soil moisture $(\theta, m^3 m^{-3})$ and vegetation cover (Van der Schalie et al., 2016; 2017) on the $h$. $\overline{\tau}_v$ is an estimate of the vegetation density based on $\tau_v$ retrieved by calculating a primary LPRM run with $A_v$ and $B_v$ set to 1 and 0, with preferably a smoothing of ± 10 days applied to the $\tau_v$ to remove noise from the signal. The minimum $h$ in LPRM is set to $h_1(B_v \overline{\tau}_v)$.
$$h= h_1(A_v(1-2 \theta)+B_v\overline{\tau}_v) \quad Eqn. 6$$

r_s is the surface reflectivity and p1 and p2 are opposite polarization (horizontal (H) or vertical (V)). The surface reflectivity is calculated from the Fresnel equations:

$$r_{s,H} = \abs{\frac{\cos u - \sqrt{\varepsilon - \sin^2u}}{\cos u + \sqrt{\varepsilon - \sin^2u}}}^2 \quad Eqn. 7$$

$$r_{s,V} = \abs{\frac{\varepsilon \cos u - \sqrt{\varepsilon - \sin^2u}}{\varepsilon \cos u + \sqrt{\varepsilon - \sin^2u}}}^2 \quad Eqn. 8$$

Where r_s,H is the horizontal polarized reflectivity, and r_s,V is the vertical polarized reflectivity and ε the complex dielectric constant of the soil surface (ε = ε' + ε"i)._ The dielectric constant is an electrical property of matter and is a measure of the response of a medium to an applied electric field. The dielectric constant is a complex number, containing a real (ε') and imaginary (ε") part. The real part determines the propagation characteristics of the energy as it passes upward through the soil, while the imaginary part determines the energy losses (Schmugge et al. 1986). There is a large contrast in dielectric constant between water and dry soil, and several dielectric mixing models have been developed to describe the relationship between soil moisture and dielectric constant (Dobson et al. 1985; Mironov et al. 2004; Peplinski et al. 1995; Wang and Schmugge 1980). Owe and Van de Griend (1998) compared the Dobson and Wang and Schmugge model and they concluded that the Wang and Schmugge model had better agreement with the laboratory dielectric constant measurements. Consequently, LPRM uses the Wang and Schmugge model, which requires information on the soil porosity (P) and wilting point (WP), observation frequency (F), T_S, and θ.

A special characteristic of LPRM is the internal analytical approach to solve for the vegetation optical depth, τ_v (Meesters et al. 2005). This unique feature reduces the required vegetation parameters to one, the single scattering albedo. LPRM makes use of the Microwave Polarization Difference Index (MPDI) to calculate τ_v. The MPDI is defined as:

$$MPDI = \frac{T_{b\_s,V} - T_{b\_s,H}}{T_{b\_s,V} + T_{b\_s,H}} \quad Eqn. 9$$

When one assumes that τ and ω have minimal polarization dependency at satellite scales, then the vegetation optical depth can be described as: 

$$\tau_v = \cos u \ ln(ad + \sqrt{(ad)^2 + a +1}) \quad Eqn. 10$$

Where

$$a = \frac{1}{2} \left[ \frac{e_{r,V} - e_{r,H}}{MPDI} - e_{r,V} - e_{r,H} \right] \quad Eqn. 11$$

And

$$d = \frac{1}{2} \frac{\omega}{(1- \omega)} \quad Eqn. 12$$

By using all these equations in combination with the dielectric mixing model, soil moisture could be solved in a forward model together with a parameterization of the following parameters: atmosphere, soil and vegetation temperature (T_a, T_s, T_c), the optical depth of the atmosphere (τ_a), the roughness parameters Q and h, soil wilting point (WP) and porosity (P), and the single scattering albedo (ω).

The temperatures were estimated using Ka-band (37 GHz) observations according to the method of Holmes et al. (2009).

For the day-time (ascending) observations the following equation is used:

$$T_S = 0.898T_{b\_37V} + 44.2 \quad Eqn. 13$$

and for the night-time (descending):

$$T_S = 0.893T_{b\_37V} + 44.8 \quad Eqn. 14$$

However, since the current L-band missions do not observe the Earth at the Ka-band frequency, they still require modelled T_S from land surface models as an input. Van der Schalie et al. (2021) have developed a method to replace the land surface model with data from an inter-calibrated dataset (ICTB) based on six passive microwave sensors. However, this is not yet applied to L-band data used in C3S SM.

The soil P and WP were derived from the FAO soil texture map available from the Harmonised World Soil Database (FAO/IIASA/ISRIC/ISS-CAS/JRC, 2009), while all the other parameters were given a fixed value. Table 2 summarizes the values used for the different frequencies.

Table 2: Values of the different parameters used in LPRM for the different frequencies

C-, X- and Ku-band Model Parametrization

Soil moisture retrieval using higher frequencies are known to have issues in the more extreme climates, e.g. tropical regions, deserts and boreal forests. A single global parameterization function is used for L-band but leads to questionable behavior for some of the higher frequencies. For example, unnaturally high/low soil moisture values over certain land cover classes or non-valid model retrievals. Starting from LPRMv7 onwards, a variable parameterization is used. This counts for two parameters, dT (a local bias correction of the land surface temperature) and ⍵ (the single scattering albedo).

Due to more dynamic VOD behavior as compared to L-band, the VOD based vegetation correction has too much impact on the soil moisture signal from C-, X- and Ku-band. Therefore, we have studied ways to remove this effect from LPRM for the higher frequencies. Together with the renewed parameterization, the roughness has now been simplified to the soil moisture (SM) and Porosity (P) dependent:

$$h=0.5 \ast (P-SM)/P \quad Eqn. 15$$

with h = 0 when SM > P.

Prior to the optimization of the parameters, the temperature relation with Ka-band has been revisited for use in LPRMv7. Local slope and intercept have been calculated between Ka-band and the average of the ERA5-Land land surface temperature (0 cm) and ERA5-Land layer 1 soil temperature (0-7cm), see Figure 3. This dataset is known to have a good quality and the expected seasonal behavior.

Figure 3: Slope (SLP), Intercept (ITC) and Correlation (R) of Ka-band against ERA5-Land temperature. Note that the interpolation over the tropical region in Africa has since been improved.

On top of this new local temperature relation, the dT is used in the optimization to correct the effective temperature from the microwave emission, assuming that the seasonal dynamics are similar to that of ERA5-Land. During testing, a variable slope has also been considered, but this gave a negligible effect on the final results. The optimization process for both parameters (simultaneously) searches for the best correlation against SMAP L-band LPRM and limits the results to datasets that had at least 90% valid retrievals between 0.01 and 0.75 m³m^-3. Resulting parameterization can be found in Figure 4, with an example of the final quality: the standard deviation and the 10/90 percentiles, in Figure 5.

Figure 4: The single scattering albedo(⍵) and local bias correction of the land surface temperature (dT), results from the optimization for C- (6.9 GHZ, top row), X- (10.7 GHZ, middle row) and Ku-band (18.7 GHZ, bottom row).

Figure 5: The 10/90 percentiles (P10/P90) and standard deviation (STDEV) of the LPRMv7 soil moisture retrievals from C- (6.9 GHZ, top row), X- (10.7 GHZ, middle row) and Ku-band (18.7 GHZ, bottom row). Note that Greenland is normally not run and therefore shows artefacts that are not in the final product.

Day-time Retrievals

Since the temperature distribution within the vegetation and soil is not in equilibrium during the 1:30 am observation period, and the situation varies throughout the seasons, a standard approach of optimizing LPRM left too many regions without a functional parameterization. Therefore, an alternative was found that corrects the day-time brightness temperatures to night-time values and is applied to all frequencies including Ka-band for the temperature retrieval. This was done using the following steps: 

1. Linearly interpolated dataset for day-time and night-time to allow sufficient overlap.
2. Calculate the median ratio between day-time and night-time over a 3-week (±10 day) window. The median was chosen over the mean to reduce the impact of strong individual events between overpasses.
3. Apply this ratio to correct the day-time data to night-time.
4. Apply LPRM to the dataset using the exact same parameterization as for night-time retrievals.

The resulting skill of the day-time retrievals from AMSR2 using LPRMv7.0 can be seen in Figure 6. The day-time values are still reaching good (r > 0.6) correlations over much of the same regions where the night-time performs well against SMAP Level 4 SM. Because the theoretical issue of non-existing thermal equilibrium for midday observations remains present within the vegetation and soil surface, an increase in overall noise within the day-time datasets does lead to an average decrease in correlation, therefore, when available, the night-time retrievals still have a preference.

Figure 6: Visualized correlation of night-time (descending: Desc.) and day-time (ascending: Asc.) observations against SMAP Level 4 SM, for C- (6.9 GHZ, top row), X- (10.7 GHZ, middle row) and Ku-band (18.7 GHZ, bottom row) from AMSR2, using LPRMv7.0.

Known Limitations

The known limitations in deriving soil moisture from passive microwave observations are listed and described in detail in this section. These issues do not only apply to the current C3S CDR and ICDR soil moisture dataset releases but also to soil moisture retrieval from passive microwave observations in general.

Vegetation

Vegetation affects the microwave emission, and under a sufficiently dense canopy the emitted soil radiation will become completely masked by the overlying vegetation. The simultaneously derived vegetation optical depth can be used to detect areas with excessive vegetation, of which the boundary varies with observation frequency.

Figure 7 gives an example of the relationship between the analytical error estimate in soil moisture as described in the previous section and vegetation optical depth. This figure shows larger error values in the retrieved soil moisture product for higher frequencies at similar vegetation optical depth values. For example, for a specific agricultural crop (VOD=0.5), the error estimate for the soil moisture retrieval in the C-band is around 0.07 m³·m⁻³; in the X-band, this is around 0.11 m³·m⁻³, and in the Ku-band, this is around 0.16 m³·m⁻³. All relevant frequency bands show an increasing error with increasing vegetation optical depth. This is consistent with theoretical predictions, which indicate that, as the vegetation biomass increases, the observed soil emission decreases, and therefore, the soil moisture information contained in the microwave signal decreases (Owe et al., 2001).

In addition, retrievals from the higher frequency observations (i.e., X- and Ku-bands) show adverse influence by a much thinner vegetation cover. Soil Moisture retrievals with a soil moisture error estimate beyond 0.2 m³m^-3 are considered to be unreliable and are masked out. 

Figure 7: Error of soil moisture as related to the vegetation optical depth for 3 different frequency bands (from Parinussa et al., 2011).

For the L-band based retrievals from SMOS, the vegetation influence is less as compared to the C-, X- and Ku-band retrievals, which can be seen from the R_value and TCA results in Figure 8 (top). In Figure 8 (bottom), the SMOS LPRM and AMSR-E LPRM (based on C-band) are included and show more stable results over dense vegetation, i.e. Normalized Difference Vegetation Index (NDVI) values of over 0.45.

Figure 8: Triple collocation analysis (TCA: top) and R-value results (bottom) for several soil moisture datasets, including SMOS LPRM and AMSR-E LPRM, for changing vegetation density (NDVI). Based on Van der Schalie et al. (2018).

Frozen surfaces and snow

Under frozen surface conditions the dielectric properties of the water changes dramatically. As snow cover, ice, and frozen conditions were demonstrated to have a big impact on data quality and availability within the current Passive product, a uniform satellite driven flagging strategy was designed by Van der Vliet et al. (2020). Prior to this, all pixels where the surface temperature is observed to be at or below 274.15 K are assigned with an appropriate frozen data flag, this was determined using the method of Holmes et al. (2009). However, as this methodology is insufficiently accurate in detecting the transition to snow and frozen conditions, the new methodology was introduced by Van der Vliet et al. (2020), which uses three frequencies (Ku-, K- and Ka-band) to properly flag these conditions.

Barren Grounds and Desert Areas

Very dry conditions above deserts and other barren areas lead to subsurface scattering phenomena and complicate the process of defining a correct land surface temperature due to the increased sensing depth. To account for errors in very dry soils affecting the passive retrieval, flagging of these conditions is applied. Figure 9(b) illustrates how barren grounds can likely be flagged in a similar manner as the snow/frozen conditions (Van der Vliet et al., 2020). Based on MODIS Landcover data a first classification was made in LPRMv7 that was stepwise improved by considering the spatial patterns. The related decision tree can be found in Figure 9(a). 

Figure 9: (b) Barren ground and related desert regions are clearly visible when using the first step of the snow/frozen flag without the correction for low physical temperatures. (a) Final decision tree for the barren soil flagging.

The dynamic behavior of barren ground conditions is captured well, as displayed in Figure 10.

Figure 10: Example of resulting flags for four different months in the year.

Water bodies

Water bodies within the satellite footprint can strongly affect the observed brightness temperature due to the high dielectric properties of water. Especially when the size of a water body changes over time, they can dominate the signal. LPRM uses a 5 % water body threshold based on MODIS observations and pixels with more than 5 % surface water are masked (Owe et al., 2008).

Rainfall

Rainstorms during the satellite overpass can strongly affect the brightness temperature observations, and therefore should be flagged in LPRM. Ongoing investigations are done to define a proper filtering mechanism derived from the passive microwave observations themselves. Currently, only strong events are removed due to its effect on the retrieved temperature from Ka-band, which then drops below 274.15K.

Radio Frequency interference

Natural emission in several low frequency bands is affected by artificial sources, so called Radio Frequency Interference (RFI). As a diagnostic for possible errors, an RFI index is calculated according to De Nijs et al. (2015). Most passive microwave sensors that are used for soil moisture retrieval observe in several frequencies. This allows LPRM to switch to higher frequencies in areas affected by RFI.

The new methodology that is now used for RFI detection uses the estimation of the standard error between two different frequencies. It uses both the correlation coefficient between two observations and the individual standard deviation to determine the standard error in Kelvin. A threshold value of 3 Kelvin is used to detect RFI. This method does not produce false positives in extreme environments and is more sensitive to weak RFI signals in relation to the traditional methods (e.g. Li et al., 2004). 

As the currently integrated SMOS mission does not have multiple frequencies to apply this method, here we base the filtering on the RFI probability information that is supplied by in the SMOS Level 3 data. SMAP, by using different channels around 1.4GHz, already has an internal mitigation of RFI that removes almost all occurrence of RFI, therefore no extra filtering is needed for use with the C3S SM.

Updated temperature input from Ka-band observations

The land surface temperature plays a unique role in solving the radiative transfer model and therefore directly influences the quality of the soil moisture retrievals. The current linear regression to link Ka-band measurements to the effective soil temperature has been adjusted and optimized by Parinussa et al. (2016) for day-time observations (Figure 11). This is done using an optimisation procedure for soil moisture retrievals through a quasi-global precipitation-based verification technique, the so-called R_value metric. In this optimisation, different biases were locally applied to the existing linear regression and final results have been used to create an updated global linear regression. The focus of this study was to improve the skill to capture the temporal dynamics of the soil moisture. After the updated linear regression for the land surface temperature, the R_value increased on average with 16.5% and the triple collocation analysis showed an average reduction in Root Mean Square Error (RMSE) of 15.3%, showing an improved skill in day-time retrievals from LPRM.

Figure 11: (a) comparison of R-value with the old and new day-time land surface temperature binned over NDVI, (b) the difference in R-value compared to the old temperature parameterisation in [%].

This explorative work showed the high impact of temperature on the quality of the LPRM retrievals. Secondly, there are issues with (seasonal) bias in the effective temperature derived from Ka-band using a linear regression method, which are caused by the seasonal changes in soil moisture, vegetation cover and atmospheric composition. The day-time retrieval has been finally integrated with LPRM as of version 7.0, according to the methodology detailed in Section 3.1.4.

In order to remove model dependency for the L-band soil moisture retrievals, we have collocated Ka-band observations from other satellites to SMAP and SMOS. This way, temperature input to the L-band missions is taken from actual satellite observations. This also allows to apply the new filtering methodology developed by Van der Vliet et al. (2020).

Active Microwave Algorithm

The active microwave soil moisture products utilized in the generation of the C3S soil moisture datasets are obtained from external operational sources as follows:

• ERS-1 AMI surface soil moisture products have been generated at TU Wien (2013) utilizing change detection method described in Wagner et al. 1999b.
• ERS-2 AMI surface soil moisture data sets stem from reprocessing activities which have been carried out within ESA's SCIRoCCo project (Crapolicchio et al., 2016). The ERS-2 data set used in all ESA CCI SM versions is the ERS.SSM.H.TS 25 km soil moisture time series product (ESA, 2017).
• Metop ASCAT surface soil moisture data sets stem from the EUMETSAT Satellite Application Facility on Support to Operational Hydrology and Water Management (H SAF²).

A detailed overview of the retrieval methodology employed in the generation of the Surface Soil Moisture products from the ERS-1 AMI instrument is provided in Wagner et al. 1999b, for the Surface Soil Moisture products from the ERS-2 AMI instrument please refer to Crapolicchio et al. 2004, and for the ASCAT derived products an overview of the retrieval methodology can be found in H-SAF 2018c.

Merging Strategy

This chapter is largely based upon the “Algorithm Theoretical Baseline Document (ATBD) D2.1 Supporting Version 08.1” [D1] document produced under ESA Climate Change Initiative Soil Moisture project.

Principle of the merging process

The generation of the long-term soil moisture data set involves three steps (as detailed in Figure 12): (1) merging the original passive microwave soil moisture products into one product, (2) merging the original active microwave soil moisture products into one product, and (3) merging all original active and passive microwave soil moisture products into one. The input datasets considered for generating the merged soil moisture product are:

• Scatterometer-based soil moisture products
  • Sensors: ERS-1/2 and Metop-A/B/C ASCAT
  • Retrieval method: TU Wien change detection method (Wagner et al. 1999b; H-SAF 2018c)
  • Time span: 1991 – now
• Radiometer-based soil moisture products
  • Sensors: SMMR, SSM/I, TRMM, AMSR-E, AMSR2, WindSat, SMOS, SMAP, FengYun 3B/C/D, GPM
  • Retrieval method: VUA-NASA LPRM v6 and v7 model (Owe et al. 2008; van der Schalie et al. 2016)
  • Time span: 1978 – now
• Modelled 0 – 10 cm soil moisture from the Noah land surface model of the Global Land Data Assimilation System version 2.0 and v2.1 (GLDAS; Rodell et al., 2004).
  • Time span: 1948 – 2010 and 2000 - Present (0.25-degree resolution)

The homogenized and merged product presents surface soil moisture with a global coverage and a spatial resolution of 0.25°. The time period spans the entire period covered by the individual sensors, i.e. 1978 – now, while measurements are provided at a 1-day sampling.

Figure 12: Overview of the three-step blending approach from original products to the final blended active & passive microwave soil moisture product (Adapted from Liu et al. 2012)

Overview of processing steps

The level 2 surface soil moisture products derived from the active and passive remotely sensed data undergo a number of processing steps in the merging procedure (see Figure 13 for an overview):

1. Spatial resampling and temporal resampling (including flagging and cross-flagging of observations)
2. Rescaling passive and active level 2 observations into radiometer and scatterometer climatologies (for the PASSIVE and ACTIVE product), and separately rescaling all level 2 observations into a common model-based climatology (for the COMBINED product)
3. Triple collocation analysis (TCA)-based error characterisation of all rescaled level 2 products. This step is performed on a seasonality basis, meaning that variations in uncertainties of different sensors within a year (e.g., due to vegetation dynamics) are captured.
4. Polynomial regression between VOD and error estimates to fill spatial gaps where errors could not be reliably retrieved i.e., where TCA is deemed unreliable
5. Merging rescaled passive and active time series into the PASSIVE, ACTIVE, and COMBINED products, respectively. This step is performed on a seasonality basis, meaning that intra-annually varying biases are corrected.
6. Break correction algorithm applied to the merged COMBINED product (Preimesberger et al., 2021)

Figure 13: Overview of the processing steps in the C3S product generation: The merging of two or more data sets is done by weighted averaging and involves overlapping time periods, whereas the process of joining data sets only concatenates two or more data sets between the predefined time periods. *The [SSM/I, TMI] period is specified not only by the temporal, but also by the spatial latitudinal coverage.

Methodology

In this section, the algorithms of the scaling and merging approach are described. Notice that several algorithms, e.g. rescaling, are used in various steps of the process, but will be described only once.

Spatial Resampling

The sensors used for the different merged products have different technical specifications (Table 3, Table 4). Obvious are the differences in spatial resolution and crossing times. Both elements need to be brought into a common reference before the actual merging can take place.

The final merged products are provided on a regular grid with a spatial resolution of 0.25° in both latitude and longitude extension. This is a trade-off between the higher resolution scatterometer data and the generally coarser passive microwave observations without leading to any under-sampling. The same resolution is often adopted by land surface models.

For the LPRM passive data, nearest neighbor resampling is performed on the radiometer input data sets to bring them into the common regular grid. Following this resampling technique, each grid point in the reference (regular grid) data set is assigned to the value of the closest grid point in the input dataset. In general, the nearest neighbor resampling algorithm can be applied to data set with regular degree grid.

For the active microwave data sets, where equidistant grid points are defined by the geo-reference location of the observation, the hamming window function is used to resample the input data to a 0.25° regular grid. The search radius is a function of latitude of the observation location, as the distance between two regular grid points reduces as the location tends towards the poles. In contrast, the active microwave data set uses the discrete global grid (DGG), where the distance between every two points is the same. This main difference between the DGG (active) and the targeted regular degree grid is rectified by using a hamming window with search radius dependent on the latitude for the spatial resampling of the active microwave data.

Table 3: Major characteristics of passive microwave instruments used C3S SM

(1) For passive microwave instruments, this stands for the footprint spatial resolution.
(2) LPRM v6.1 references: van der Schalie et al. (2016, 2017, 2018), van der Vliet (2020)
(3) LPRM v6.2 consists of a temporal extension of LPRMv6.1, including day-time retrievals.
(4) Data from SSM/I F08, F11, F13 satellites are used
Table 4: Major characteristics of active microwave instruments and model products used in C3S SM

(1) For active microwave instruments, this stands for the footprint spatial resolution.
(2) TU Wien change detection algorithm references for AMI-WS: Wagner et al. (1999)

Temporal Resampling

The temporal sampling of the merged product is 1 day. The reference time for the merged dataset is set at 0:00 UTC. For each day starting from the time frame center at 0:00 UTC observations within ±12 hours are considered.

The temporal resampling strategy first searches for the valid observation that is closest to the reference time. In the case that there are only invalid observations, which are flagged other than "0" (zero), within this time frame, the closest measurement among these invalid observations is selected. If there are no measurements available at all within a time frame, no value is assigned to that day. This strategy results in data gaps when no observations within ±12 hours from the reference time are available. 

Starting from v202212, day-time observations are available from the passive sensors (LPRMv7). While it is planned in future dataset versions to achieve a sub-daily temporal resolution, the temporal resampling at the current stage applies indistinctly to day- and night-time observations.

Flagging

During the temporal resampling stage, flagging is applied to the datasets where relevant information is available. The key flags set during this process are 'frozen', 'high VOD' and 'Other' and these flags are propagated through the entire processing chain to the final product.

The ASCAT and ERS products include a Surface State Flag (SSF) which effectively encodes information about whether or not the surface is frozen or snow-covered. In the ESA CCI SM / C3S SM product, those soil moisture values where Bit 1 (indicating frozen conditions) in the binary representation of the SSF bitflag is active are used to flag the observation as frozen. The ASCAT and ERS products do not provide information on high VOD.

The LPRMv6.1 (and following) products provide a FLAGS field which provides information on high VOD, frozen conditions, and the performance of the LPRM algorithm. The thresholds above which VOD is considered 'high' are set based on the saturation point in the VOD signal for each sensor and band. This is the point at which the VOD value is considered to equal 100% vegetation signal. Secondly, the frozen/snow flag was applied using the new approach by Van der Vliet et al. (2020), which derives the frozen/snow conditions from Ku-, K- and Ka-band observations. In addition to this, from LPRMv7 onwards a flag for barren grounds and desert areas is included (see section 3.1.5.3). This is implemented as an advisory flag as of C3S SM v202212. Including it as a critical flag would have a big impact on the soil moisture availability in desert-prone regions. Therefore, more validation is needed to establish whether the optional usage of the barren ground flag or a more conservative version thereof (as a critical flag) is suitable in C3S SM. As of v202312, the barren ground flag is only activated at locations / times where the majority of available L3 observations are classified respectively. 

A cross-flagging approach is implemented for frozen soils. This means that any frozen flags provided in any of the datasets are effectively transferred to all of the datasets. It works by reading in all of the flag data for all of the datasets, determining if the frozen flag is set in any of them and if it is, applying it to all relevant observations in all of the sensors. 

Rescaling

Due to different observation frequencies, observation principles, and retrieval techniques, the contributing soil moisture datasets are available in different observation spaces. Therefore, before merging can take place at either level, the datasets need to be rescaled into a common climatology. All soil moisture observations of each product are rescaled to the climatology of a different reference, namely AMSRE, ASCAT or GLDAS for the passive, active or combined product respectively.

Scaling is performed by using CDF matching which is a well-established method for calibrating datasets with deviating climatologies (Drusch et al. 2005; Liu et al. 2007; Liu et al. 2011; Reichle et al. 2004, Moesinger et al. 2020). CDF-matching is applied for each grid point individually and based on piece-wise linear matching. This variation of the CDF-matching technique proved to be robust also for shorter time periods (Liu et al. 2011). The matching is shown by means of an example for a grid point centred at 41.375 ^oN, 5.375 ^oW. Figure 14 shows for this location the time series of soil moisture estimates from GLDAS-Noah, AMSR-E and ASCAT, respectively. CDF-matching for this time series is performed in the following way:

1. For the time-collocated data points, CDFs (Figure 15) are computed. In the passive product AMSR2 is scaled using the parameters derived from the last 3 years of AMSRE and first 3 years of AMSR2. SMAP is then scaled to the scaled AMSR2.
2. If more than 400 time-collocated data points exist, for each CDF curve the 0, 5, 10, 20, 30, 40, 50, 60, 70, 80, 90, 95 and 100 percentiles are identified. Else evenly spaced percentile bins are generated such that each of them contains at least 20 observations.
3. Use the n percentiles of the CDF curves to define n-1 segments. The CDF curves of these circled values are shown in Figure 15(a), (b) and (c).

   $$slope_i = \frac{pref_{i+1} - pref_i}{psrc_{i+1} - psrc_i} \quad Eqn. 16$$ $$intercept_i = pref_i - (psrc_i \ast slope_i) \quad Eqn. 17$$

   where 𝑖=1…12, is the number of the segments, and 𝑝𝑟𝑒𝑓 is the percentile of the GLDAS-Noah data (reference), and 𝑝𝑠𝑟𝑐 is the percentile of either AMSR-E or ASCAT data (source) respectively.
4. An exception is the first and the last segment. Instead of using the first and last percentile for interpolation, the slope is derived using least squares regression. This is more robust to outliers.
5. The n percentile values from the AMSR-E and ASCAT CDF curves are plotted against those of Noah (Figure 15(d) and (e)) and scaling linear equations (e.g., slope and intercept) between two consecutive percentiles are computed.
6. The obtained linear equations are used to scale all observations of the target data set (i.e., also the time steps that do not have a corresponding observation in the reference data set) to the climatology of the reference data set (Figure 15(f)).

   $$sm_r = slope_i \ast sm + intercept_i \quad Eqn. 18$$

   where sm_r is the rescaled soil moisture and sm is the original soil moisture value. slope_i and intercept_i are chosen depending on the sm value and its corresponding i-percentile. The AMSR-E and ASCAT values outside of the range of CDF curves can also be properly rescaled, using the linear equation of the closest value.

Figure 14: Time series of soil moisture estimates from (a) Noah, (b) AMSR-E and (c) ASCAT for a grid cell (centred at 41.375° N, 5.375° W) in 2007. Circles represent days when Noah, AMSR-E and ASCAT all have valid estimates. (Figure taken from Liu et al. 2011)

Figure 15: Example illustrating how the cumulative distribution function (CDF) matching approach was implemented to rescale original AMSR-E and ASCAT against Noah soil moisture product in this study. (a, b, c) CDF curves of AMSR-E, Noah and ASCAT soil moisture estimates for the grid cell shown in Fig. 2. (d) Linear regression lines of AMSR-E against Noah for 12 segments. (e) Same as (d), but for ASCAT and Noah. (f) CDF curves of Noah (black), rescaled AMSR-E (blue) and rescaled ASCAT (red) soil moisture products. (Figure taken from Liu et al. 2011)

Intra-annual bias correction

From version v202212 on, the source and reference samples used in the scaling procedure are first divided into 366 subsets corresponding to the data points belonging to each day of the year. The scaling parameters are then calculated and applied separately to each subset, effectively providing a seasonality and thus accounting for non-stationary biases between the different sensors (or between a sensor and the reference model). These biases are generated by the different effect that the seasonally variable environmental conditions exert on the retrieval. One of such effects is for instance the vegetation state which impacts differently the scatterometric and radiometric datasets.

As the time series are subset into smaller samples, it can occur that the calculation of scaling parameters relies on too few observations, leading to overfitting. To avoid this, the usual threshold of a minimum of 20 points is used, below which the parameters are not calculated. In this case, the global time series parameters are used, allowing to prevent data loss.

Rescaling of Active Datasets

Different sensor specifications between ERS1/2 and ERS2 (e.g. spatial resolution) need to be compensated for using scaling. The CDF curves for ERS2 are calculated based on the overlap with ERS1/2. Rescaling ERS2 against ERS1/2 and then joining them generates the AMI-WS active data set, which is subsequently scaled to the Metop-A ASCAT data (ACTIVE product) or the GLDASv2.1 data (COMBINED product) (see Figure 13).

For the ACTIVE product, the limited overlap between AMI-WS ERS1/2 and Metop-A ASCAT in time (i.e., a few months) rules out the global adjustment method based on the information of their overlapping period. However, as retrievals from Metop-A ASCAT and AMI-WS capture similar seasonal cycles (Liu et al., 2011), we assume that their dynamic ranges are identical and, therefore, can use non-overlapping observations for the rescaling (i.e. the entire time period for each sensor).

It was noticed that the soil moisture (and backscatter) signal from Metop-B ASCAT is characterized by a positive bias on a global level. This has a detrimental effect particularly on the soil moisture trends from the ACTIVE product. To correct for this, Metop-B ASCAT is scaled to the reference of Metop-A ASCAT leading to a homogenization of the data record, as shown in Figure 16.

Figure 16: Comparison of global and hemispheric averages of soil moisture from ASCAT before (left) and after rescaling of Metop-B ASCAT on Metop-A ASCAT.

Rescaling of Passive Datasets

The seasonal cycle associated with the SSM/I dataset is deemed to be unreliable and therefore, for all CCI products, the SSM/I seasonal cycle is replaced with that from AMSR-E. The high frequency variations (anomalies) associated with SSM/I are scaled to those from AMSR-E prior to recombining the decomposed signal. An example of the SSM/I decomposition and rescaling is shown in Figure 17.

For the PASSIVE product, all datasets with the exception of SMAP are rescaled to AMSR-E. Where sufficient overlap is available, this is utilised; for all other cases (except AMSR2), the entire time period of AMSR-E and the sensor being scaled is utilised. For AMSR2, data in the last three years of AMSR-E and the first three years of ASMR2 are used, i.e. 2008-10-04 to 2015-07-01. SMAP is rescaled to AMSR2 which has already been rescaled to AMSR-E.

Figure 17: Example illustrating how (a) the TMI was rescaled against AMSR-E, (b-e) the SSM/I anomalies were rescaled against AMSRE-E anomalies, reconstructed and merged with rescaled TMI and AMSR-E, and (f) the SMMR was rescaled and merged with the others. The grid cell is centred at 13.875°N, 5.875°W (Image courtesy Liu et al., 2012).

Rescaling in the COMBINED product

For generating the combined product, all passive and active level 2 data sets are rescaled against GLDASv2.1, with the exception of ERS1/2, ASCAT and SSMI which are discussed above.

Error characterization

Errors in the individual active and passive products are characterized by means of triple collocation analysis. These errors are used both for estimating the merging parameters and for characterizing the errors of the merged product (see section 3.3.6).

Triple collocation analysis is a statistical tool that allows estimating the individual random error variances of three data sets without assuming any of them acting as supposedly accurate reference (Gruber et al. 2016a, 2016b). This method requires the errors of the three data sets to be uncorrelated, therefore triplets always comprise of (i) an active data set, (ii) a passive data set, and (iii) the GLDAS-Noah land surface model, which are commonly assumed to fulfil this requirement (Dorigo et al. 2010). Error variance estimates are obtained as:

$$\begin{align}
\sigma_{\varepsilon_a} &= \sigma_a^2 - \frac{\sigma_{ap} \sigma_{am}}{\sigma_{pm}} \\
\sigma_{\varepsilon_p} &= \sigma_p^2 - \frac{\sigma_{pa} \sigma_{pm}}{\sigma_{am}}
\end{align} \quad Eqn. 19$$

where σ_ε² denotes the error variance; σ² and σ denote the variances and covariances of the data sets; and the superscripts denote the active (a), the passive (p), and the modelled (m) data sets, respectively. For a detailed derivation see Gruber et al. (2016b).

The error estimates detailed above represent the average random error variance of the entire considered operational time period of a sensor, which is commonly assumed to be stationary. The soil moisture uncertainties of the three products (ACTIVE, PASSIVE, and COMBINED) have been determined until v202212 through the above equations, which caused the uncertainty estimates provided with the product to apply to an entire merging period and not vary with every (observation) timestamp. However, a seasonal characterisation of the error is fundamental to understand the error structure in EO-based soil moisture estimates and its coupling with temporally dynamic scene characteristics (Zwieback et al., 2018) and can ultimately benefit in wide variety of studies where observation reliability is highly valued, spanning from model assimilation to seasonal yield forecasts. For this reason, a new approach is used as of v202312, where TCA is performed using moving temporal subsets as also described in Chapter 3.3.3.4. Each subset comprise all the observations included in a 3-month window centred on each month of the year, therefore providing an error estimate that is representative of the relative month. The comparison between a static and seasonal approach is illustrated below (Figure 18).

Figure 18: Comparison between the static uncertainty (v202212 and before) and the seasonal uncertainty (monthly resolution, v202312 and following) characterization with TCA for ASCAT and AMSR2. On the background, VOD from VODCA C-Band (Moesinger et al., 2020) is shown.

As expected, the error structure and the relative performance between the two sensors is reflecting the seasonal vegetation patterns (Dorigo et al., 2010). The relative weights of the sensors change accordingly throughout the year, as given by Eqn. 21-22.h3.

Error gap-filling

TCA does not provide reliable error estimates in all regions, mainly if there is no significant correlation between all members of the triplet, which often happens for example in high-latitude areas or in desert areas. TCA error estimates are therefore disregarded where the Pearson correlation between any of the data sets is deemed insignificant (i.e., p-value < 0.05).

In these areas, error estimates are derived by deriving a Signal-to-Noise Ratio (SNR)-VOD regression model per land cover class and using this to determine the SNR based on the VOD at each location where SNR could not be retrieved:

$$SNR_x = \sum_{i=0}^N a_i VOD_x^i \quad Eqn. 20$$

Where the subscript denotes the spatial location; and the parameters ai are derived from a global polynomial regression between VOD and TCA based error estimates at locations where they are considered reliable (i.e., all data sets are significantly correlated). For TMI and WINDSAT, third order polynomials (N=3) are used and for all other sensors second order polynomials (N=2) are used, which was empirically found to provide the best regression results.

Merging

The merging procedure consists of (1) merging the original passive microwave product into the PASSIVE product, (2) merging the original active microwave products into the ACTIVE product, and (3) merging the original active and passive microwave products into the COMBINED product. The merging periods are shown in Figure 1 and listed in Table 5, Table 6, Table 7.

Note that in C3S SM only daytime retrievals are used for all operational passive sensors also included in the ICDR products (SMOS, SMAP, AMSR2, GPM). 

Table 5: Passive sensors used in the PASSIVE product (sensors marked with * in the last merging period are used in the ICDR).

Table 6: Active sensors used in the ACTIVE product (sensors marked with * in the last merging period are used in the ICDR).

Table 7: Sensors used in the COMBINED product (sensors marked with * in the last merging period are used in the ICDR).

Weight estimation

The merging is performed by means of a weighted average which takes into account the error properties of the individual data sets that are being merged:

$$\Theta_m = \sum_{i=1}^N w_i \cdot \Theta_i \quad Eqn. 21$$

Where Θ_m denotes the merged soil moisture product; Θ_i are the soil moisture products that are being merged, and w_i are the merging weights.
Per definition, the optimal weights for a weighted average are determined by the error variances of the input data sets and write as follows:

$$w_i = \frac{\sigma_{\varepsilon_i}^{-2}}{\sum_{j=1}^N \sigma_{\varepsilon_j}^{-2}} \quad Eqn. 22$$

where j denote the index of the respective data sets; i is the data set for which the weight is being calculated; and N is the total number of data sets which are being averaged. The required error variances are calculated using Eqn. 19. Notice that error covariance terms are neglected as they cannot be estimated reliably.

It should be mentioned that the above definition of the weights based on error variances assumes all data sets to be in the same data space. However, data sets usually vary in their signal variability due to algorithmic differences, varying signal frequencies, etc. Therefore, conceptually, it is more appropriate to define relative weights in terms of the data sets SNR properties rather than of their error variance (Gruber et al., 2017). Nevertheless, the actual merging requires a harmonization of the data sets into a common data space, which in the case of the CCI SM data set is done using the CDF matching approach described in Section 3.3.3.4 . Therefore, the calculation of the weights using Eqn. 22 suffices, keeping in mind that they represent the rescaled error variances of rescaled data sets.

Notice that soil moisture estimates of the various sensors are not available every day, hence there are certain dates during the overlapping periods on which not all data sets provide a valid estimate to calculate the weighted average. In such cases, the weights are re-distributed amongst the remaining data sets, again based on their relative SNR properties.

However, this re-distribution of weights could significantly worsen data quality on these days because of the increasing contribution of measurements which initially would have had a low weight due to their (relatively) low SNR. Therefore, soil moisture estimates in the merged product on days where not all data sets provide valid estimates are set to NaN values (Not a Number), if the sum of the initial weight of the remaining data sets is lower than 1/(2N) where N is the total number of data sets that are potentially available for the corresponding merging period. This threshold has been derived empirically to provide a good trade-off between temporal measurement density and average data quality. 

Similar to the generation of the PASSIVE product, relative weights at each time step are derived from the TCA- or VOD-regression based error estimates for each individual sensor. Depending on how many sensors are available within a particular period, a (1/2N) threshold for the minimum weight of a particular sensor was applied if not all sensors provide a soil moisture estimate at that day.

Break detection and correction

In C3S v202312, break detection and correction methods are introduced to the COMBINED product. These methods use reanalysis soil moisture as a reference to detect inhomogeneities in the merged satellite records.

Breaks may occur as a result of merging different sensor combinations over time, as shown in Figure 19. Such breakpoints may therefore appear between periods with different input sensors. Structural inhomogeneities may affect statistics such as trends and changes in extreme values (percentiles) and therefore should not only be detected but also corrected.

Figure 19: Potential break times in the ESA CCI SM v04.4 (COMBINED) product corresponding to changeovers in the blended sensors, building the homogeneous (sensor) sub-periods (HSP).

Based on the work of Su et al. (2016), a procedure has been developed at TU Wien to test for potential inhomogeneities in the ESA CCI SM CDR using the Fligner-Killeen test for homogeneity of variances and Wilcoxon rank-sums test for shifts in population mean ranks (Preimesberger et al., 2021). For the product provided in v202312, ERA5 is used as the reference dataset.

To adjust detected breaks in the data set, Quantile Category Matching (QCM) is used. This method uses split-fitted differences in empirical CDFs of the candidate and reference SM values (between quantile categories, i.e. average SM within a number of quantile ranges) before and after a break. The values after the break are used to find corrections for quantiles of satellite measurements before the break.

Adjustment is performed iteratively, with the goal that across each detected break, changes in C3S SM means and variances are matched to follow changes within the reference data set (relative bias correction) and homogenized observation series (with respect to the reference data set) are derived.

The results of the correction performed on v04.4 of the ESA CCI SM dataset are shown in Figure 20. Figure 21 shows the longest homogenous period, i.e. the maximum period over which no break was detected, of available data both before (top) and after (bottom) correction using the QCM method. 

Figure 20: Results of the inhomogeneity testing (between HSP₃ and HSP₄) before any correction methods have been applied (top) with the results of the testing after the QCM correction method is applied (bottom). Adapted from (Preimesberger et al., 2021)

Figure 21: Longest homogenous period in ESA CCI SM v04.4 (COMBINED) before adjustment (top) and after adjustment (bottom) using the QCM method. Taken from (Preimesberger et al., 2021).

Near real time (NRT) mode of the processor

The processing steps described in the previous sections work on the complete time series of the datasets. This is not feasible during NRT production of the products. Figure 22 gives an overview of the NRT processing chain. The parameters for CDF scaling and the characterised errors are precomputed during generation of the CDR and then used directly in the ICDR processing. This speeds up the processing and results in a consistent time series based on stable merging parameters. At the moment it is only possible to use night-time retrievals from passive sensors in the ICDR generation.

Figure 22: Overview of NRT processing chain

Output data

Table 8 lists the output fields generated by the processing system and available to data users. Detailed information about each field and the file format are provided in the Product User Guide and Specification (PUGS) [D3].

Table 8: Overview of output data fields

References

Ashcroft, P. and Wentz, F. (2000). Algorithm Theoretical Basis Document: AMSR Level-2A Algorithm, Revised 03 November. Santa Rosa, California USA: Remote Sensing Systems. https://eospso.gsfc.nasa.gov/sites/default/files/atbd/atbd-amsr-level2A.pdf (URL resource last accessed 18th December 2023)  

Attema, E., & Ulaby, F. (1978). Vegetation modeled as water cloud. Radio Science, 13, 357-364

Balsamo, G., Albergel, C., Beljaars, A., Boussetta, S., Brun, E., Cloke, H., Dee, D., Dutra, E., Muñoz-Sabater, J., Pappenberger, F., de Rosnay, P., Stockdale, T., and Vitart, F. (2015), “ERA-Interim/Land: a global land surface reanalysis dataset”, Hydrol. Earth Syst. Sci., vol. 19, pp. 389-407, doi: 10.5194/hess-19-389-2015.

Balsamo, G., S. Boussetta, P. Lopez, L. Ferranti (2010), Evaluation of ERA-Interim and ERA-Interim-GPCP-rescaled precipitation over the U.S.A., ERA Report Series, n. 5, pp10 [pdf].

Bartalis, Z., Wagner, W., Naeimi, V., Hasenauer, S., Scipal, K., Bonekamp, H., Figa, J., & Anderson, C. (2007). Initial soil moisture retrievals from the METOP-A Advanced Scatterometer (ASCAT). Geophy. Res. Lett, 34, L20401

Beaudoing, H. and Rodell M., NASA/GSFC/HSL (2016), GLDAS Noah Land Surface Model L4 3 hourly 0.25 x 0.25 degree V2.1, Greenbelt, Maryland, USA, Goddard Earth Sciences Data and Information Services Center (GES DISC), Accessed: [Data Access Date], 10.5067/E7TYRXPJKWOQ

Bennartz, R., and Michelson, D.B. (2012), Validation of AMSR-E and AMSU/HSB level 1 brightness temperatures and level 2 cloud and precipitation parameters, NAG5--12579, Meteorological and Hydrological Institute, Report, March 2004 https://pubs.ssec.wisc.edu/research_Resources/publications/pdfs/METPUBS/MET_Publication_No_04_03_B1.pdf (URL resource last accessed 18th December 2023) 

Bevington, P.R., & Robinson, D.K. (2002). Data reduction and error analysis for the physical sciences. (3th ed.). Boston: McGraw-Hill Science/Engineering/Math

Bevis, M., Businger, S., Herring, T., Rocken, C., Anthes, R., & Ware, R. (1992). GPS Meteorology: Remote Sensing of Atmospheric Water Vapor Using the Global Positioning System. Journal of Geophysical Research, 97, null-15801

CATDS SMOS L3 soil moisture retrieval processor Algorithm Theoretical Baseline Document (ATBD) (2013), https://labo.obs-mip.fr/wp-content-labo/uploads/sites/18/2013/08/ATBD_L3_rev2_draft.pdf

CATDS LEVEL 3 DATA PRODUCT DESCRIPTION Soil Moisture and Brightness Temperature (2014), https://www.cen.uni-hamburg.de/en/icdc/data/land/docs-land/so-tn-cb-ca-0001-3a.pdf (URL resource last accessed 18th December 2023)

Chan, S., Dunbar, R. S. (2020), Soil Moisture Active Passive (SMAP) Mission Level 3 Passive Soil Moisture Product Specification Document Version 7.0. online: https://nsidc.org/sites/default/files/psd_spl3smp_v7.pdf (URL resource last accessed 18th December 2023)

Crapolicchio, R., A. Bigazzi, G. De Chiara, X. Neyt, A. Stoffelen, M. Belmonte, W. Wagner, C. Reimer (2016) The scatterometer instrument competence centre (SCIRoCCo): Project's activities and first achievements, Proceedings European Space Agency Living Planet Symposium 2016, 9-13 May 2016, Prague, Czech Republic, 9-13.

Crow, W.T., Wagner, W., & Naeimi, V. (2010). The impact of radar incidence angle on soil moisture retrieval skill. IEEE Geoscience and Remote Sensing Letters, 7, 501-505

De Jeu R.A.M, T.R.H. Holmes, R. M. Parinussa, M Owe (2014), A spatially coherent global soil moisture product with improved temporal resolution, Journal of Hydrology 516, 284-296.

De Nijs, A.H., Parinussa, R.M., De Jeu, R.A.M., Schellekens, J. and Holmes, T.R. (2015), “A methodology to determine radio-frequency interference in AMSR2 observations”, IEEE Transactions on Geoscience and Remote Sensing, vol. 53, no. 9, pp. 5148-5159.

Dee, D.P. et al., 2011. The ERA-Interim reanalysis: configuration and performance of the data assimilation system. Quarterly Journal of the Royal Meteorological Society, 137(656), pp.553–597

Dobson, M.C., Ulaby, F.T., Hallikainen, M.T., & Elrayes, M.A. (1985). Microwave dielectric behavior of wet soil. Part II: Dielectric mixing models. IEEE Transactions on Geoscience and Remote Sensing, 23, 35-46

Dorigo, W.A., et al. (2011) "The International Soil Moisture Network: A data hosting facility for global in situ soil moisture measurements", Hydrology and Earth System Sciences 15 (5), pp. 1675-1698

Dorigo, W.A., Scipal, K., Parinussa, R.M., Liu, Y.Y., Wagner, W., De Jeu, R.A.M., and Naeimi, V. (2010), “Error characterisation of global active and passive microwave soil moisture datasets”, Hydrology and Earth System Sciences, vol. 14, pp. 2605 – 2616, doi: 10.5194/hess-14-2605-2010.

Dorigo, W., Wagner, W., Albergel, C., Albrecht, F., Balsamo, G., Brocca, L., Chung, D., Ertl, M., Forkel, M., Gruber, A., et al. (2017), “ESA CCI Soil Moisture for improved Earth system understanding: state-of-the art and future directions”, Remote Sensing of Environment, https://doi.org/10.1016/j.rse.2017.07.001

Dorigo, W., Himmelbauer, I., Aberer, D., Schremmer, L., Petrakovic, I., Zappa, L., Preimesberger, W., Xaver, A., Annor, F., Ardö, J., Baldocchi, D., Bitelli, M., Blöschl, G., Bogena, H., Brocca, L., Calvet, J.-C., Camarero, J. J., Capello, G., Choi, M., Cosh, M. C., van de Giesen, N., Hajdu, I., Ikonen, J., Jensen, K. H., Kanniah, K. D., de Kat, I., Kirchengast, G., Kumar Rai, P., Kyrouac, J., Larson, K., Liu, S., Loew, A., Moghaddam, M., Martínez Fernández, J., Mattar Bader, C., Morbidelli, R., Musial, J. P., Osenga, E., Palecki, M. A., Pellarin, T., Petropoulos, G. P., Pfeil, I., Powers, J., Robock, A., Rüdiger, C., Rummel, U., Strobel, M., Su, Z., Sullivan, R., Tagesson, T., Varlagin, A., Vreugdenhil, M., Walker, J., Wen, J., Wenger, F., Wigneron, J. P., Woods, M., Yang, K., Zeng, Y., Zhang, X., Zreda, M., Dietrich, S., Gruber, A., van Oevelen, P., Wagner, W., Scipal, K., Drusch, M., and Sabia, R.: The International Soil Moisture Network: serving Earth system science for over a decade, Hydrol. Earth Syst. Sci., 25, 5749–5804, https://doi.org/10.5194/hess-25-5749-2021 (URL resource last accessed 18th December 2023), 2021.

Drusch, M., Wood, E., & Gao, H. (2005). Observation operators for the direct assimilation of TRMM microwave imager retrieved soil moisture. Geophysical Research Letters, 32, L15403

Entekhabi, D., Das, N., Njoku, E.G.,Yueh, S., Johnson, J., Shi, J.(2014) Algorithm Theoretical Basis Document L2 & L3 Radar/Radiometer Soil Moisture, L2 & L3 Radar/Radiometer Soil Moisture, 

ESA (2017) ERS-2 SCATTEROMETER Surface Soil Moisture Time Series in High Resolution - ERS.SSM.H.TS (25 km Time-Series product), SCI-MAN-16-0047-v02 https://earth.esa.int/eogateway/documents/20142/37627/scirocco-pum-ts.pdf  (URL resource last accessed 18th December 2023)

FAO/IIASA/ISRIC/ISS-CAS/JRC, 2009. Harmonized World Soil Database (version 1.1). FAO, Rome, Italy and IIASA, Laxenburg, Austria

Fu, C. C., D. Han, S. T. Kim, and P. Gloersen (1988). User's guide for the Nimbus- 7 Scanning Multichannel Microwave Radiometer (SMMR) CELL-ALL tape. NASA Reference Publication #1210, National Aeronautics and Space Administration, Washington, D.C.    

Fung, A.K. (1994). Microwave scattering and emission models and their applications. Boston: Artech House

Gaiser, P.W., K. M. St. Germain, E. M. Twarog, G. A. Poe, W. Purdy, D. Richardson, W. Grossman, W. L. Jones, D. Spencer, G. Golba, J. Cleveland, L. Choy, R. M. Bevilacqua and P. S. Chang (2004), "The WindSat spaceborne polarimetric microwave radiometer: Sensor description and early orbit performance," IEEE Trans. Geosci. Remote Sens., vol. 42, 2347-2360, http://www.emc.ncep.noaa.gov/mmb/data_processing/satellite_ingest.doc/File_Format.doc/windsatdocjan06-1.pdf (URL resource last accessed 18th December 2023)

Gloersen, P., D. J. Cavalieri, A. T. C. Chang, T. T. Wilheit, W. J. Campbell, O. M. Johannessen, K. B. Katsaros, K. F. Kunzi, D. B. Ross, D. Staelin, E. P. L. Windsor, F. T. Barath, P. Gudmansen, E. Langham, and R. Ramseier (1984). A summary of results from the first Nimbus-7 SMMR observations. J. Geophys. Res. 89, 5335-5344.

GPM Science Team (2022), GPM GMI XCAL Common Calibrated Brightness Temperatures L1BASE 1.5 hours 13 km V07, Greenbelt, MD, USA, NASA Goddard Earth Science Data and Information Services Center (GES DISC), 10.5067/GPM/GMI/BASE-XCAL/07

Gruber, A., Dorigo, W., Crow, W., and Wagner, W. (2017), “Triple Collocation-Based Merging of Satellite Soil Moisture Retrievals”, IEEE trans. geosc. rem. sens., DOI: 10.1109/TGRS.2017.2734070

Gruber, A., Su, C.H., Crow, W.T., Zwieback, S., Dorigo, W.A., & Wagner, W. (2016a). Estimating error cross-correlations in soil moisture data sets using extended collocation analysis. Journal of Geophysical Research: Atmospheres, 121(3), 1208-1219.

Gruber, A., Su, C.H., Zwieback, S., Crow, W.T., Wagner, W., & Dorigo, W. (2016b). Recent advances in (soil moisture) triple collocation analysis. International Journal of Applied Earth Observation and Geoinformation, 45, 200-211.

Gruber, A., De Lannoy, G., Albergel, C., lL-yaari, A., Brocca, L., Calvet, J. C., Colliander, A., Cosh, M., Crow, W., Dorigo, W., Draper, C., Hirschi, M., Kerr, Y., Konings, A., Lahoz, W., Mccoll, K., Montzka, C., Munoz-Sabater, J., Peng, J., Reichle, R., Richaume, P., Rüdiger, C., Scanlon, T., Van Der Schalie, R., Wigneron, J. P. & Wagner, W. (2020). Validation practices for satellite soil moisture retrievals: What are (the) errors? Remote Sensing of Environment, 244, 111806.

Hersbach, H, Bell, B, Berrisford, P, et al. The ERA5 global reanalysis. Q J R Meteorol Soc. 2020; 146: 1999–2049. https://doi.org/10.1002/qj.3803 (URL resource last accessed 18th December 2023)H SAF (2018a). ASCAT Surface Soil Moisture CDR2017 time series 12.5 km sampling – Metop (H113), EUMETSAT SAF on Support to Operational Hydrology and Water Management. http://dx.doi.org/10.15770/EUM_SAF_H_0005 (URL resource last accessed 18th December 2023)

H SAF (2018b). ASCAT Surface Soil Moisture CDR2017-EXT time series 12.5 km sampling – Metop (H114), EUMETSAT SAF on Support to Operational Hydrology and Water Management. https://navigator.eumetsat.int/product/EO:EUM:DAT:0108?query=H114&s=advanced (URL resource last accessed 18th December 2023)  

H SAF (2018c) Algorithm Theoretical Baseline Document (ATBD), Metop ASCAT Soil Moisture Data Records v0.7 (H113)  https://hsaf.meteoam.it/CaseStudy/GetDocumentUserDocument?fileName=ASCAT_SSM_CDR_ATBD_v0.7.pdf&tipo=ATBD (URL resource last accessed 18th December 2023)  

Hillburn KA, and CL Shie, (2011) Decadal trends and variability in special sensor microwave imager (SSM/I) Brightness Temperatures and Earth Incidencs angle, NASA RSS Technical Report 092811

Holmes, T.R.H., De Jeu, R.A.M., Owe, M., & Dolman, A.J. (2009). Land surface temperature from Ka band (37 GHz) passive microwave observations. Journal of Geophysical Research -Atmospheres, 114

Hsieh, C.-Y., Fung, A.K., Nesti, G., Sieber, A.J., & Coppo, P. (1997). A further study of the IEM surface scattering model. IEEE Transaction on Geoscience and Remote Sensing, 35, 901-909

Kerr, Y.H., Waldteufel, P., Wigneron, J.P., Delwart, S., Cabot, F., Boutin, J., Escorihuela, M.J., Font, J., Reul, N., Gruhier, C., Juglea, S.E., Drinkwater, M.R., Hahne, A., Martin-Neira, M., and Mecklenburg, S. (2010), “The SMOS mission: New tool for monitoring key elements of the global water cycle”, Proceedings of the IEEE, vol. 98, no. 5, doi: 10.1109/JPROC.2010.2043043.

Kottek, M., Grieser, J., Beck, C., Rudolf, B., & Rubel, F. (2006). World Map of the Köppen-Geiger climate classification updated. Meteorologische Zeitschrift, 15, 259-263

Lehner, B. & Döll, P., 2004. Development and validation of a global database of lakes, reservoirs and wetlands. Journal of Hydrology, 296(1-4), pp.1–22

Li, L., Njoku, E.G., Im, E., Chang, P.S., and St. German, K. (2004), “A preliminary survey of radio-frequency interference over the U.S. in Aqua AMSR-E data”, IEEE Trans. Geosc., vol. 42, no. 2, pp. 380 – 390, doi: 10.1109/TGRS.2003.817195.

Liu, Y., de Jeu, R.A.M., van Dijk, A.I.J.M., & Owe, M. (2007). TRMM-TMI satellite observed soil moisture and vegetation density (1998-2005) show strong connection with El Nino in eastern Australia. Geophysical Research Letters, 34, Art. No. L15401

Liu, Y.Y., Dorigo, W.A., Parinussa, R.M., De Jeu, R.A.M., Wagner, W., McCabe, M.F., Evans, J.P., & Van Dijk, A.I.J.M. (2012). Trend-preserving blending of passive and active microwave soil moisture retrievals. Remote Sensing of Environment, 123, 280-297

Liu, Y.Y., Parinussa, R.M., Dorigo, W.A., de Jeu, R.A.M., Wagner, W., van Dijk, A., McCabe, F.M., & Evans, J.P. (2011). Developing an improved soil moisture dataset by blending passive and active microwave satellite-based retrievals. Hydrology and Earth System Sciences, 15, 425-436

Meesters, A., De Jeu, R.A.M., & Owe, M. (2005). Analytical derivation of the vegetation optical depth from the microwave polarization difference index. IEEE Geoscience and Remote Sensing Letters, 2, 121-123

Mo, T., Choudhury, B.J., Schmugge, T.J., Wang, J.R., & Jackson, T.J. (1982). A model for microwave emission from vegetation-covered fields. Journal of Geophysical Research-Oceans and Atmospheres, 87, 1229-1237

Mironov, V.L., Dobson, M.C., Kaupp, V.H., Komarov, S.A., & Kleshchenko, V.N. (2004). Generalized refractive mixing dielectric model for moist soils. IEEE Transactions on Geoscience and Remote Sensing, 42, 773-785

Moesinger, L., Dorigo, W., de Jeu, R., van der Schalie, R., Scanlon, T., Teubner, I., and Forkel, M.: The global long-term microwave Vegetation Optical Depth Climate Archive (VODCA), Earth Syst. Sci. Data, 12, 177–196, https://doi.org/10.5194/essd-12-177-2020 (URL resource last accessed 18th December 2023), 2020.

Muñoz-Sabater, J., Dutra, E., Agustí-Panareda, A., Albergel, C., Arduini, G., Balsamo, G., Boussetta, S., Choulga, M., Harrigan, S., Hersbach, H., Martens, B., Miralles, D. G., Piles, M., Rodríguez-Fernández, N. J., Zsoter, E., Buontempo, C., and Thépaut, J.-N.: ERA5-Land: a state-of-the-art global reanalysis dataset for land applications, Earth Syst. Sci. Data, 13, 4349–4383, https://doi.org/10.5194/essd-13-4349-2021 (URL resource last accessed 18th December 2023), 2021.

Naeimi, V., Bartalis, Z., and Wagner, W., (2009). ASCAT soil moisture: An assessment of the data quality and consistency with the ERS scatterometer heritage. Journal of Hydrometeorology, 10(2), pp.555-563.

Newell, D., Draper, D.,Figgins, D., Berdanier, B., Kubitschek, M., Holshouser, D., Sexton, A. , Krimchansky, S., Wentz, F., Meissner, T., "GPM microwave imager key performance and calibration results," 2014 IEEE Geoscience and Remote Sensing Symposium, 2014, pp. 3754-3757, doi: 10.1109/IGARSS.2014.6947300.

Njoku, E. (1996). Nimbus-7 SMMR Pathfinder brightness temperatures. Boulder, CO: National Snow and Ice Data Center 

Njoku, E. G., B. Rague, And K. Flemming (1995): Nimbus-7 Scanning Multichannel Microwave Radiometer (SMMR): Brightness Temperature Data (SMMR Level 1B Pathfinder). JPL Publication, Jet Propulsion Laboratory, Pasadena, CA. 'User's Guide for the Scanning Multichannel Microwave Radiometer Instrument First Year Antenna Temperature Data Set', Systems & Applied Sciences Corporation, August 1982, NASA Contract NAS5-27393. 'Nimbus-7 SMMR Pathfinder Brightness Temperatures', NSIDC Dataset Guide Documentation.

O'Neill, P. E., S. Chan, E. G. Njoku, T. Jackson, R. Bindlish, and J. Chaubell. 2020a. SMAP L3 Radiometer Global Daily 36 km EASE-Grid Soil Moisture, Version 7. Boulder, Colorado USA. NASA National Snow and Ice Data Center Distributed Active Archive Center. doi: https://doi.org/10.5067/HH4SZ2PXSP6A (URL resource last accessed 18th December 2023)

O'Neill, P. E., R. Bindlish, S. Chan, J. Chaubell, E. Njoku, and T. Jackson. 2020b. SMAP Algorithm Theoretical Basis Document: Level 2 & 3 Soil Moisture (Passive) Data Products, Revision F, August 31, 2020, SMAP Project, JPL D-66480, Jet Propulsion Laboratory, Pasadena, CA. (see https://nsidc.org/sites/nsidc.org/files/technical-references/L2_SM_P_AR_R17_Aug2020_clean_final.pdf, URL resource last accessed 18th December 2023)

O’Neill, P. E., S. Chan, R. Bindlish, M. Chaubell, A. Colliander, F. Chen, S. Dunbar, T. Jackson, J. Peng, M. Cosh, T. Bongiovanni, J. Walker, X. Wu, A. Berg, H. McNairn, M. Thibeault, J. Martínez-Fernández, Á. González-Zamora, E. Lopez-Baeza, K. Jensen, M. Seyfried, D. Bosch, P. Starks, C. Holifield Collins, J. Prueger, Z. Su, R. van der Velde, J. Asanuma, M. Palecki, E. Small, M. Zreda, J. Calvet, W. Crow, Y. Kerr, S. Yueh, and D. Entekhabi. 2020c. Calibration and Validation for the L2/3_SM_P Version 7 and L2/3_SM_P_E Version 4 Data Products, SMAP Project, JPL D-56297, Jet Propulsion Laboratory, Pasadena, CA. (see https://nsidc.org/sites/nsidc.org/files/technical-references/L2_SM_P_AR_R17_Aug2020_clean_final.pdf, URL resource last accessed 18th December 2023).

Owe M., R. de Jeu and J. Walker, "A methodology for surface soil moisture and vegetation optical depth retrieval using the microwave polarization difference index," in IEEE Transactions on Geoscience and Remote Sensing, vol. 39, no. 8, pp. 1643-1654, Aug. 2001. doi: 10.1109/36.942542

Owe, M., de jeu, R., & Holmes, T. (2008). Multisensor historical climatology of satellite-derived global land surface moisture. Journal of Geophysical Research-Earth Surface, 113, F01002

Owe, M., & Van de Griend, A.A. (1998). Comparison of soil moisture penetration depths for several bare soils at two microwave frequencies and implications for remote sensing. Water Resour. Res., 34, 2319-2327

Parinussa, R, TRH Holmes and RAM de Jeu (2012), Soil moisture retrievals from the WindSat spaceborne polarimetric microwave radiometer. IEEE Trans. Geosci. Remote Sens., DOI: 10.1109/TGRS.2011.2174643

Parinussa, R.M., Meesters, A., Liu, Y.Y., Dorigo, W., Wagner, W., & de Jeu, R.A.M. (2011). Error Estimates for Near-Real-Time Satellite Soil Moisture as Derived From the Land Parameter Retrieval Model. IEEE Geoscience and Remote Sensing Letters, 8, 779-783

Parinussa, R. M., R. H. Thomas, N. W. Holmes, A. D. Wouter, A. M. Richard, and R. A. de Jeu. (2014). “A Preliminary Study toward Consistent Soil Moisture from AMSR2.” Journal of Hydrometeorology 16 (2): 932–947. doi:10.1175/JHM-D-13-0200.1

Parinussa RM, De Jeu RAM, Van der Schalie R, Crow WT, Lei F, Holmes TRH. A Quasi-Global Approach to Improve Day-Time Satellite Surface Soil Moisture Anomalies through the Land Surface Temperature Input. Climate. 2016; 4(4):50. https://doi.org/10.3390/cli4040050 (URL resource last accessed 18th December 2023)

Peplinski, N.R., Ulaby, F.T., & Dobson, M.C. (1995). Dielectric properties of soils in the 0.3-1.3 GHz range. IEEE Transactions on Geoscience and Remote Sensing, 33, 803-807

Preimesberger, W., Scanlon, T., Su, C. -H., Gruber, A. and Dorigo, W., Homogenization of Structural Breaks in the Global ESA CCI Soil Moisture Multisatellite Climate Data Record, in IEEE Transactions on Geoscience and Remote Sensing, vol. 59, no. 4, pp. 2845-2862, April 2021, doi: 10.1109/TGRS.2020.3012896

Reichle, R.H., Koster, R.D., Dong, J., & Berg, A.A. (2004). Global Soil Moisture from Satellite Observation, Land Surface Models, and Ground Data: Implications for Data Assimilation. Journal of Hydrometeorology, 5, 430-442

Rodell, M. et al., 2004. The Global Land Data Assimilation System. Bulletin of the American Meteorological Society, 85, pp.381–394.

Rui, H., Beaudoing, H., Loeser, C. and Li, B. (2018). README Document for NASA GLDAS Version 2 Data Products. [online] Hydro1.gesdisc.eosdis.nasa.gov. Available at: https://hydro1.gesdisc.eosdis.nasa.gov/data/GLDAS/README_GLDAS2.pdf. (URL resource last accessed 18th December 2023).

Schmugge, T.J., O'Neill, P.E., & Wang, J.R. (1986). Passive microwave soil moisture research. IEEE Transaction on Geoscience and Remote Sensing, GE-24, 12-22

Schneeberger, K., Schwank, M., Stamm, C., de Rosnay, P., Matzler, C., & Fluhler, H. (2004). Topsoil structure influencing soil water retrieval by microwave radiometry. Vadose Zone Journal, 3, 1169-1179

Seto, S.; Takahashi, N.; Iguchi, T. Rain/No-Rain Classification Methods for Microwave Radiometer Observations over Land Using Statistical Information for Brightness Temperatures under No-Rain Conditions. J. Appl. Meteorol. 2005, 44, 1243–1259.

Su, C.-H., Ryu, D., Dorigo, W., Zwieback, S., Gruber, A., Albergel, C., Reichle, R. H., and Wagner, W. (2016), Homogeneity of a global multisatellite soil moisture climate data record, Geophys. Res. Lett., 43, 11,245–11,252, doi:10.1002/2016GL070458.

TU Wien (2013). ERS AMI-WS (ESCAT) Surface Soil Moisture Product generated from E1/2-SZ-WNF/UWI-00 dataset. Department of Geodesy and Geoinformation, TU Wien, 2013.

Ulaby, F.T., Moore, B., & Fung, A.K. (1981). Microwave Remote Sensing - Active and Passive, Vol. I: Fundamentals of Radiometry. Addison-Weslay, 56 pp

Ulaby, F.T., Moore, B., & Fung, A.K. (1982). Microwave Remote Sensing - Active and Passive, Vol. II: Radar Remote Sensing and Surface Scattering and Emission Theory. Norwood: Artech

Uppala, S.M. et al., 2005. The ERA-40 re-analysis. Quarterly Journal of the Royal Meteorological Society, 131(612), pp.2961–3012.

Van der Schalie, R., De Jeu, R.A.M., Kerr, Y.H., Wigneron, J.-P., Rodriguez-Fernandez, N.J., Al-Yaari, A., Parinussa, R.M., Mecklenburg, S., and Drusch, M. (2017), “The merging of radiative transfer based surface soil moisture data from SMOS and AMSR-E”, Remote Sensing of Environment, vol. 189, doi: http://dx.doi.org/10.1016/j.rse.2016.11.026 (URL resource last accessed 18th December 2023).

Van der Schalie, R., Kerr, Y.H., Wigneron, J.P., Rodriguez-Fernandez, N.J., Al-Yaari, and De Jeu, R.A.M. (2016), “Global SMOS Soil Moisture Retrievals from The Land Parameter Retrieval Model”, Int. J. Appl. Earth Observ. Geoinf., doi: http://dx.doi.org/10.1016/j.jag.2015.08.005 (URL resource last accessed 18th December 2023).

Van der Schalie, R., Parinussa, R.M., De Jeu, R.A.M., Kerr, Y.H., Wigneron, J.P., Rodriguez-Fernandez, N.J., Al-Yaari, A., Mecklenburg, S., and Drusch, M. (2018), “The Effect of Three Different Data Fusion Approaches on the Quality of Soil Moisture Retrievals from Multiple Passive Microwave Sensors”, Remote Sensing, vol. 10 (1), no. 107, doi: https://doi.org/10.3390/rs10010107 (URL resource last accessed 18th December 2023).

van der Schalie R, van der Vliet M, Rodríguez-Fernández N, Dorigo WA, Scanlon T, Preimesberger W, Madelon R, de Jeu RAM. L-Band Soil Moisture Retrievals Using Microwave Based Temperature and Filtering. Towards Model-Independent Climate Data Records. Remote Sensing. 2021; 13(13):2480. https://doi.org/10.3390/rs13132480 (URL resource last accessed 18th December 2023).

Van der Vliet, Mendy, Robin van der Schalie, Nemesio Rodriguez-Fernandez, Andreas Colliander, Richard de Jeu, Wolfgang Preimesberger, Tracy Scanlon, and Wouter Dorigo (2020). Reconciling Flagging Strategies for Multi-Sensor Satellite Soil Moisture Climate Data Records. Remote Sensing 12, no. 20: 3439.  

Wagner, W. (1998). Soil Moisture Retrieval from ERS Scatterometer Data. In, Institute for Photogrammetry and Remote Sensing. Vienna: Technical University of Vienna

Wagner, W., Lemoine, G., Borgeaud, M., & Rott, H. (1999a). A Study of Vegetation Cover Effects on ERS Scatterometer Data. IEEE Transactions on Geoscience and Remote Sensing, 37, 938-948

Wagner, W., Lemoine, G., & Rott, H. (1999b). A Method for Estimating Soil Moisture from ERS Scatterometer and Soil Data. Remote Sensing of Environment, 70, 191-207

Wagner, W., Noll, J., Borgeaud, M., & Rott, H. (1999c). Monitoring soil Moisture over the Canadian Prairies with the ERS Scatterometer. IEEE Trans. Geosci. Rem. Sens., 37, 206-216

Wagner, W., & Scipal, K. (2000). Large-Scale Soil Moisture Mapping in western Africa using the ERS Scatterometer. IEEE Trans. Geosci. Rem. Sens., 38, 1777-1782

Wang, J.R., & Choudhury, B.J. (1981). Remote sensing of soil moisture content over bare field at 1.4 GHz frequency. Journal of Geophysical Research-Oceans and Atmospheres, 86, 5277-5282

Wang, J.R., & Schmugge, T.J. (1980). An empirical model for the complex dielectric permittivity of soils as a function of water content. IEEE Transactions on Geoscience and Remote Sensing, 18, 288-295

Wentz, F. J. (1991), User's Manual: SSM/I Antenna Temperature Tapes (Revision 1), report number 120191, Remote Sensing Systems, Santa Rosa, CA, 73 pp. (available online: https://images.remss.com/papers/rsstech/1991_120191_Wentz_SSMI_TA_manual_rev1.pdf, URL resource last accessed 18th December 2023)

Wentz, F. J. (1993), User's Manual: SSM/I Antenna Temperature Tapes (Revision 2), report number 120193, Remote Sensing Systems, Santa Rosa, CA, 36 pp. (available online: https://images.remss.com/papers/rsstech/1993_120193_Wentz_SSMI_TA_manual_rev2.pdf, URL resource last accessed 18th December 2023)

Wentz, F. J., L. Ricciardulli, K. A. Hilburn, and C. A. Mears (2007), How much more rain will global warming bring?, Science, 317, 233-235.

WindSat Data Products Users’ Manual, Sensor & Environmental Data Records, Version 3.0, January 2006, D- 29825, JPL.

WMO (2016). The Global Observing System for Climate (GCOS): Implementation Needs GCOS-200 Available online: https://library.wmo.int/doc_num.php?explnum_id=3417(URL resource last accessed 18th December 2023).

Xu, Z., Qian, J., Xian, D., Qi, Y. "Fengyun-3 Series Meteorological Satellite Data Archiving and Service System," 2016 IEEE International Geoscience and Remote Sensing Symposium (IGARSS), 2016, pp. 5477-5480, doi: 10.1109/IGARSS.2016.7730427.

Yang, H., Zou, X., Li, X.,You, R., "Environmental Data Records From FengYun-3B Microwave Radiation Imager," in IEEE Transactions on Geoscience and Remote Sensing, vol. 50, no. 12, pp. 4986-4993, Dec. 2012, doi: 10.1109/TGRS.2012.2197003.

Yang, H., Weng, F., Lv, L., Lu, N., Liu, G., Bai, M. Qian, Q., He, J., Xu, H., "The FengYun-3 Microwave Radiation Imager On-Orbit Verification," in IEEE Transactions on Geoscience and Remote Sensing, vol. 49, no. 11, pp. 4552-4560, Nov. 2011, doi: 10.1109/TGRS.2011.2148200.

Zwieback, S., Colliander, A., Cosh, M. H., Martínez-Fernández, J., McNairn, H., Starks, P. J., Thibeault, M., and Berg, A.: Estimating time-dependent vegetation biases in the SMAP soil moisture product, Hydrol. Earth Syst. Sci., 22, 4473–4489, 2018.

Related articles