Last modified on Jun 27, 2024 17:45

Contributors: Richard de Jeu (vandersat), Robin van der Schalie (vandersat/planet labs), Christoph Paulik (vandersat), Wouter Dorigo (tuwien), Tracy Scanlon (tuwien), Adam Pasik (tuwien), Richard Kidd (EODC), Christoph  Reimer (EODC)

Issued by: EODC/Richard Kidd

Date: 02/06/2020

Ref: C3S_312b_Lot4.D1.SM.2-v2.0_202001_Algorithm_Theoretical_Basis_Document_v1.0

Official reference number service contract: 2018/C3S_312b_Lot4_EODC/SC2

History of modifications

List of datasets covered by this document

Related documents

Click here to expand the list of related documents (D1-D2)

Reference ID	Document
D1	T. Scanlon, A. Pasik., W. Dorigo, R.A.M de Jeu, S. Hahn, R. van der Schalie, W. Wagner, R. Kidd, A. Gruber, L. Moesinger., W. Preimesberger (2019), "Algorithm Theoretical Baseline Document (ATBD) D2.1 Version 04.7", See documents at https://www.esa-soilmoisture-cci.org/node/119
D2	Product User Guide and Specification (PUGS), Version 2.0: Soil Moisture (Deliverable ID: D3.SM.5-v2.0)

Acronyms

Scope of the document

The Algorithm Theoretical Basis Document (ATBD) provides a detailed description of the algorithms that are used within the C3S Soil Moisture production system to produce the soil moisture Climate Data Record (CDR) and Interim Climate Data Records (ICDR). This document relates to C3S product versions v201912 for CDR and ICDR products.

Executive summary

The C3S Soil Moisture production system is based on the production system initially developed within ESA’s Climate Change Initiative Soil Moisture Project (Phase 1 & 2) [D1] and has been updated and re-engineered to enable the provision of soil moisture retrievals on a near-real-time basis. In this context “near real time” refers to the provision of soil moisture information a minimum of 10 days and a maximum of 20 days after sensing. This ATBD consists of a description of the input and auxiliary data used in the soil moisture retrievals, a description of the algorithms behind the soil moisture retrievals for both active and passive microwave observations, a description of the merging process and finally an overview of the output data.

More specifically this document relates to the C3S Soil Moisture production system used to generate product versions v201912. The algorithm used to generate the CDR and ICDR remains unchanged from product version v201812 and is based on ESA CCI SM v04.4.  The current dataset (v201912) is a temporal extension of v2018012 to cover the period to 2019-12-31.

Chapter one provides an overview of the satellite instruments that are used for the production of the soil moisture data, which is a combination of both passive and active microwave sensors dating back to 1978. The second chapter presents an overview of all the auxiliary data that is used during the processing of the soil moisture dataset, for which a distinction is made between input and the validation data. The third chapter gives an in depth description of the algorithm used to retrieve soil moisture from passive microwave observations and points to relevant resources regarding active microwave retrievals from external operational sources. Chapter three also provides detailed information on how the retrievals from different satellite sensors are then combined into a consistent merged soil moisture database. The fourth and final chapter briefly describes the output fields of the final soil moisture product.

Product Change Log

The following Table, Table 1, (from section 1.6 of [D2]) provides an overview of the differences between different versions of the product up-to, and including, the current version v201912 (CDRv2.0) see List of datasets covered by this document.

Table 1: Changes in the product between versions.

1. Instruments

The CDR/ICDR each comprise of three products, being termed "active", "passive" and "combined", with each product being provided at three different temporal resolutions (daily, dekadal and monthly). A suite of data from various sensors are used in generating each product depending upon the temporal period and latitude of the retrieval. A schematic overview of the instruments used in the generation of the CDR/ICDR is provided in Figure 1, with information about the sensor characteristics being provided in sections 1.1 and 1.2.

Figure 1: Spatial-temporal coverage of input products used to construct the CDR/ICDR (a) ACTIVE, (b) PASSIVE, (c) COMBINED. Blue colours indicate passive, red colours active microwave sensors. The periods of unique sensor combinations are referred to as 'blending period'. Modified from Dorigo et al. (2017).

1.1. Passive Microwave Systems

1.1.1. Nimbus-7 SMMR (NASA)

Originating System	Scanning Multichannel Microwave Radiometer on board Nimbus 7
Data class	Earth observation
Sensor Type and key technical characteristics	Microwave Radiometer, 5 frequencies data in swath format (level 1b) 6.6, 10.7, 18 and 37 GHz in both horizontal and vertical polarization, 21 GHz in vertical polarization Inc. angle: 50.3 degrees Spatial sampling: 25x25 km Spatial resolution: 148x95 km (6.6 GHz), 91x59 km (10.7 GHz), 55x41 km (18 GHz), 46x30 km (21 GHz), 27x18 km (37 GHz) Temporal resolution, ~6 days, sun-synchronous observation with (daytime (12 hr) and night time (24 hr) equator crossings. Data Citation: Njoku 1996
Data Availability and Coverage	October 1978 - August 1987, 180°W 90°S – 180°E 90°N A summary of the data can be found on https://nsidc.org/data/nsidc-0036/versions/1 Fu et al. (1988)
Source Data Name and Product Technical Specifications	SMMR Level 1b Technical Specification Njoku, 1996 https://nsidc.org/data/nsidc-0036/versions/1 Fu et al. (1988)
Data Quantity	Total volume is 70 GB (compressed)
Data Quality and Reliability	Instrument specification: Radiometric sensitivity: 0.7 K (6.6 GHz), 0.8 K (10.7 GHz), 0.9 K (18 GHz), 1 K (21 GHz), 1.4 K (37 GHz) Validation reports Njoku et al. (1995) Gloersen et al. (1984)
Ordering and delivery mechanism	Ordering via NSIDC Data Centre http://nsidc.org Delivery is possible via FTP over Internet
Access conditions and pricing	Freely accessible
Issues	In 1986 there is a high frequency of bad antenna counts, especially during and for some time after the Special Operations Period (April – October 1986). Interpolating radiometric samples within scans and between adjacent scans tends to smear the effect of these "bad" antenna counts, which are most noticeable in browse image maps of 6.6 GHz horizontal polarization data.

1.1.2. DMSP-SSM/I (NESDIS NOAA)

1.1.3. TRMM–TMI (NASA/JAXA)

1.1.4. AQUA-AMSR-E

1.1.5. Coriolis WindSat (Naval Research Laboratory)

1.1.6. GCOM-W1 AMSR-2 (JAXA)

1.1.7. SMOS (ESA)

1.2. Active Systems

1.2.1. ERS AMI (ESA)

Originating System	Active Microwave Instrument (AMI) Wind Scatterometer (WS) on-board ERS-1 and ERS-2
Data class	Earth Observation data
Sensor Type and key technical characteristics	Scatterometer with 3 fan-beam antennae Frequency 5.3 GHz ± 52kHz (C-Band) Polarization: linear vertical (VV) Inc. angles: 18 – 47 (mid), 25 – 59 (fore/aft) Spatial resolution: ≥ 45 km Swath width: ≥ 500 km Swath stand-off: 200 km to right of sub-sat. Track
Data Availability and Coverage	IFREMER 1991/08/05 – 2001/01/17, 180°W 82°S – 180°E 82°N ESA Rolling Archive 2003/08/13 – 2010/01/28, 180°W 82°S – 180°E 82°N
Source Data Name and Product Technical Specifications	IFREMER ERS.WSC.WNF (IFREMER product WNF Format) Technical Specification: Spatial Resolution: 50x50 km Spatial Sampling: 25x25 km C2-MUT-W-01-IF ESA Rolling Archive ERS.WSC.UWI (fast delivery product BUFR Format) Technical Specification: Spatial Resolution: 50x50 km Spatial Sampling: 25x25 km https://directory.eoportal.org/web/eoportal/satellite-missions/e/ers-1
Data Quantity	~32 GB
Data Quality and Reliability	Instrument specification Radiometric stability: 0.46 dB Localisation accuracy: 5 km
Ordering and delivery mechanism	IFREMER Ordering via CERSAT webpage http://cersat.ifremer.fr, currently no information about delivery mechanism. (CD) ESA Rolling Archive Ordering via ESA Earthnet Online http://earth.esa.int Delivery is possible via FTP
Access conditions and pricing	Freely accessible
Issues	Due to the loss of gyroscopes onboard of ERS-2 in January 2001, data from 2001/01/17 to 2003/08/13 is lost.

1.2.2. ASCAT Metop-A (EUMETSAT)

1.2.3. ASCAT Metop-B (EUMETSAT)

2. Input and auxiliary data

In the following an overview of the input and auxiliary data is provided. A division is made between data which is used as input for data production (section 2.1) and for validation activities (section 2.2).

2.1. Input data

2.1.1. Advisory Flags

The following section describes datasets that are used to generate the advisory flags provided as additional information to the soil moisture products. The advisory flags are not provided as part of the soil moisture datasets but only as datasets to support interpretation and analysis of the soil moisture products.

2.1.1.1. SSM/I-SSMIS EASE-Grid Daily Global Ice Concentration and Snow Extent

2.1.1.2. ERA-40

2.1.1.3. Global lakes and wetlands database (GLWD)

Originating System	See Table 1 in Lehner and Döll (2004)
Data class	GIS database
Sensor Type and key technical characteristics	Organisation in three levels: Level 1 comprises the shoreline polygons of the 3067 largest lakes (surface area 50 km2) and 654 largest reservoirs (storage capacity 0.5 km3) worldwide, and offers extensive attribute data. Level 2 contains the shoreline polygons of approx. 250,000 smaller lakes, reservoirs and rivers (surface area 0.1 km2), excluding all water bodies of level 1 Level 3 represents lakes, reservoirs, rivers, and different wetland types in the form of a global raster map at 30-second resolution, including all water bodies of levels 1 and 2.
Data Availability and Coverage	Global (except Antarctica)
Source Data Name and Product Technical Specifications	GLWD http://www.worldwildlife.org/science/data/WWFBinaryitem8606.pdf
Data Quantity	~50 MB
Data Quality and Reliability	More information is provided in Lehner, B. & Döll, P., 2004.
Ordering and delivery mechanism	Download from FTP https://www.worldwildlife.org/pages/global-lakes-and-wetlands-database
Access conditions and pricing	GLWD is available for non-commercial scientific, conservation, and educational purposes.
Issues	N/A

2.1.1.4. GTOPO30

2.1.2. Modelled Data

2.1.2.1. Global Land Data Assimilation System (GLDAS) V2.0

2.1.2.2. Global Land Data Assimilation System (GLDAS) V2.1

2.1.2.3. ERA Interim

2.1.2.4. ERA-Interim/Land

2.2. Validation data

2.2.1. In situ data from the International Soil Moisture Network (ISMN)

3. Algorithms

3.1. Passive Microwave Algorithm

This chapter is largely based upon the “Algorithm Theoretical Baseline Document (ATBD) D2.1 Version 04.7” [D1] document produced under ESA Climate Change Initiative Soil Moisture project.

3.1.1. Principles of the Land Parameter Retrieval Model

Brightness temperatures can be derived from several passive microwave sensors with different radiometric characteristics, i.e. Nimbus SMMR, the Tropical Rainfall Measuring Mission (TRMM) Microwave Imager (TMI), Microwave Imaging Radiometer with Aperture Synthesis (MIRAS) onboard the Soil Moisture and Ocean Salinity (SMOS) mission and the Advanced Microwave Scanning Radiometer (AMSR-E) on the AQUA Earth observation satellite. The observed brightness temperatures are converted to soil moisture values with the Land Parameter Retrieval Model (LPRM, Owe et al., 2008). This model is based on a microwave radiative transfer model that links soil moisture to the observed brightness temperatures. A unique aspect of LPRM is the simultaneous retrieval of vegetation density in combination with soil moisture and surface temperature. A result of this physical parameterization is that any differences in frequency and incidence angle that exist among different satellite platforms are accounted for within the framework of the radiative transfer model based on global constant parameters (De Jeu et al., 2014). This important aspect makes LPRM suitable for the development of a long term consistent soil moisture network within ESA's CCI soil moisture project.

The different processing steps of LPRM are described in detail in the next section, while Figure 2 presents a flowchart of the entire methodology.

Figure 2: Flowchart of the main processes of the Land Parameter Retrieval Model (LPRM). Soil moisture is solved when the observed brightness temperature equals the modelled brightness temperature as derived by the radiative transfer.

3.1.2. Methodology

The thermal radiation in the microwave region is emitted by all natural surfaces and is a function of both the land surface and the atmosphere. According to LPRM the observed brightness temperature (T_b) as measured by a space borne radiometer can be described as:

Where Γ_a and Γ_v are the atmosphere and vegetation transmissivity respectively, T_{b_s} is the surface brightness temperature, e_r is the rough surface emissivity, T_{b_extra}, the extra-terrestrial brightness temperature and the T_{b_u} and T_{b_d} are the upwelling and downwelling atmospheric brightness temperatures. The subscript p denotes either horizontal (H) or vertical (V) polarization. The vegetation/atmosphere transmissivity is further defined in terms of the optical depth, τ_v/a, and satellite incidence angle, u, such that

The upwelling brightness temperature from the atmosphere is estimated as (Bevis et al. 1992):

Were T_a is the atmospheric temperature. In LPRM the downwelling Temperature (T_d) is assumed to be equal to the upwelling temperature (T_u) and the Extraterrestrial temperature is set to 2.7 K (Ulaby et al. 1982). The radiation from a land surface (T_bp) is described according to a simple radiative transfer (Mo et al. 1982):

Where T_s and T_v are the thermodynamic temperatures of the soil and the vegetation, ω is the single scattering albedo. LPRM uses the model of Wang and Choudhury (1981) to describe the rough surface emissivity as:

r_s is the surface reflectivity and p1 and p2 are opposite polarization (horizontal or vertical). The surface reflectivity are calculated from the Fresnel equations:

Where r_s,H is the horizontal polarized reflectivity, and r_s,V is the vertical polarized reflectivity and ε the complex dielectric constant of the soil surface (ε = ε' + ε"i). The dielectric constant is an electrical property of matter and is a measure of the response of a medium to an applied electric field. The dielectric constant is a complex number, containing a real (ε') and imaginary (ε") part. The real part determines the propagation characteristics of the energy as it passes upward through the soil, while the imaginary part determines the energy losses (Schmugge et al. 1986). There is a large contrast in dielectric constant between water and dry soil, and several dielectric mixing models have been developed to describe the relationship between soil moisture and dielectric constant (Dobson et al. 1985; Mironov et al. 2004; Peplinski et al. 1995; Wang and Schmugge 1980). In Owe et al. (1998) compared the Dobson and Wang and Schmugge model and they concluded that the Wang and Schmugge model had better agreement with the laboratory dielectric constant measurements. Consequently, LPRM uses the Wang and Schmugge model, which requires information on the soil porosity (P) and wilting point (WP), observation frequency (F), T_S, and θ.

A special characteristic of LPRM is the internal analytical approach to solve for the vegetation optical depth, τ_v (Meesters et al. 2005). This unique feature reduces the required vegetation parameters to one, the single scattering albedo. LPRM makes use of the Microwave Polarization Difference Index (MPDI) to calculate τ_v. The MPDI is defined as:

When one assumes that τ and ω have minimal polarization dependency at satellite scales, then the vegetation optical depth can be described as:

Where

And

By using all these equations in combination with the dielectric mixing model, soil moisture could be solved in a forward model together with a parameterization of the following parameters; atmosphere, soil and vegetation temperature (T_a, T_s, T_c), the optical depth of the atmosphere (τ_a), the roughness parameters Q and h, soil wilting point (WP) and porosity (P), and the single scattering albedo (ω).
The temperatures were estimated using Ka-band (37 GHz) observations according to the method of Holmes et al. (2009). For the day time (ascending) observations the following equation is used:

and for the night time (descending):

However, since the current L-band missions do not observe the Earth at the Ka-band frequency, they still require modelled T_S from land surface models as an input, which is something that will be improved in the near future to ensure an entirely model-independent soil moisture dataset.
The soil P and WP were derived from the FAO soil texture map available from the Harmonised World Soil Database (FAO/IIASA/ISRIC/ISS-CAS/JRC, 2009), while all the other parameters were given a fixed value. Table 2 summarizes the values used for the different frequencies.

Currently LPRMv06 is used for the processing of the soil moisture data and was already used for the full data record of SMOS, AMSRE and AMSR2. Data for SMMR, TRMM and WindSat is still based on the LPRMv05 version, but expected to be updated in the near future. For Ku-band measurements this shift in versions has no impact. The parameterization as used in LPRMv05 is also given in Table 2, with the minor different that h is a constant and the primary run is already the conclusive final run instead of having a second run with the vegetation correction as in LPRMv06. The difference in quality between the two versions can be found in Van der Schalie et al. (2017).

Table 2: Values of the different parameters used in LPRMv06 for the different frequencies.* The values for the LPRMv05 algorithm as used for SMMR, TRMM and WindSat in the historical database.

3.1.3. Analytical Derivation of the Soil Moisture Uncertainties

An uncertainty analysis for soil moisture retrievals as derived from passive microwave observations according to the Land Parameter Retrieval Model (LPRM; Owe et al., 2008) was presented in Parinussa et al. (2011). Their methodology was based on standard error propagation, as can be found in general statistical textbooks (Bevington and Robinson 2002), and provides information about how the uncertainty in each of the input parameters propagate to the soil moisture output.

where U is the partial derivative matrix. When the errors and the internal correlations between the input parameters are known, the accuracy of soil moisture can be calculated.

LPRM is a zero order radiative transfer based model. Several input parameters are affected by instrumentation uncertainties, data acquisition, reduction limitations, methodology and environmental factors. Each of these errors will introduce an uncertainty in the final soil moisture product as derived from LPRM. In general, we are not able to determine the actual error in the result if no considered true data are available for evaluation of the experimental model. Therefore, we need to develop a consistent error model for uncertainty determination. This error model informs us about the random errors, but not the biases and it does not tell us whether the model itself is correct or wrong. So the main task of an uncertainty analyses is to quantify the random error in the output of a model under the assumption that the model itself is physically correct.

Because of the high computational costs of statistical methods (e.g. Monte Carlo simulations), it's not feasible to apply such techniques on a global and (sub-) daily scale. A possible solution is proposed in the following part, where the radiative transfer equation was rewritten and an analytical solution of the quantitative uncertainty for passive microwave remote sensing of soil moisture product was derived.
The basis of the analytical solution to calculate the error in the soil moisture product lies in the use of the most basic error propagation methodology presented in most statistical textbooks; for example the function x=f(u,v,...) .

The methodology is adapted here to determine the variance σ_k² of the dielectric constant (k), using the variances of several input parameters. After the determination of the variance in the dielectric constant, a dielectric mixing model (Wang and Schmugge 1980) was used to calculate the uncertainty in soil moisture. The challenge in using the basic error propagation methodology is to define the partial derivatives.

To define the partial derivatives, we used the Jacobian matrix. The Jacobian matrix is a matrix containing the first order partial derivatives of the radiative transfer equation with respect to each variable. In our case the Jacobian matrix (J) can be described as

After applying the land surface temperature assumption, one is able to rewrite the radiative transfer equation (Parinussa et al., 2011) after putting T_L,S outside brackets, for convenience we drop subscript 'P' for polarization.

This can be rewritten to

For convenience we define the expressions F(Γ,ω) Eqn. 15 and G(Γ,ω) Eqn. 16 to rewrite equation Eqn. 12, resulting in Eqn. 13

The rough surface emissivity e_r(P) follows from Eqn. 17, wherein the horizontal (H) and vertical (V) polarization are reintroduced. This equation was written to calculate the rough surface emissivity in horizontal polarization. To calculate the rough surface emissivity at vertical polarization the (H) and (V) sign for polarization should be swapped. Q is the roughness parameter known as the cross polarization, h is the roughness and k refers to the dielectric constant.

where the last term refers to

The smooth surface emissivity was calculated using Eqn. 20 and 21, for convenience we drop subscript 's' from smooth emissivity.

where the Δ term refers to

The following derivatives will be needed

From these derivations it follows that the Jacobian matrix Eqn. 12 can be calculated analytically

From LPRM, it follows that variations in the observed parameters T_bH, T_bV, T_LS(obs) , ω_est and h_est are related to variations in the unknown model parameters Γ,k,T_LS , ω and h . Combining this with the inverse Jacobian matrix results in the following expression:

The second line in this equation holds the result:

Herein, the correlation between the errors in and is expressed in r. Figure 3 presents the global average error for AMSR-E C-band observation over 2008 resulting from the analytical error propagation analysis. It clearly shows standard deviation values below 0.06 m³m^-3 for all the dry and semi-arid regions and higher value up to 0.1 m³ m^-3 and beyond for the more densely vegetated regions.

Figure 3: Average estimated standard deviation of AMSR-E C-band soil moisture for 2008 as derived from the analytical error propagation analysis proposed by Parinussa et al., (2011).

3.1.4. Known Limitations

The known limitations in deriving soil moisture from passive microwave observations are listed and described in detail in this section. These issues do not only apply to the current C3S CDR and ICDR soil moisture dataset releases but also to soil moisture retrieval from passive microwave observations in general.

3.1.4.1. Vegetation

Vegetation affects the microwave emission, and under a sufficiently dense canopy the emitted soil radiation will become completely masked by the overlying vegetation. The simultaneously derived vegetation optical depth can be used to detect areas with excessive vegetation, of which the boundary varies with observation frequency. Figure 4 gives an example of the relationship between the analytical error estimate in soil moisture as described in the previous section and vegetation optical depth. This figure shows larger error values in the retrieved soil moisture product for higher frequencies at similar vegetation optical depth values. For example, for a specific agricultural crop (VOD=0.5), the error estimate for the soil moisture retrieval in the C-band is around 0.07 m³·m⁻³; in the X-band, this is around 0.11 m³·m⁻³, and in the Ku-band, this is around 0.16 m³·m⁻³. All relevant frequency bands show an increasing error with increasing vegetation optical depth. This is consistent with theoretical predictions, which indicate that, as the vegetation biomass increases, the observed soil emission decreases, and therefore, the soil moisture information contained in the microwave signal decreases (Owe et al., 2001). In addition, retrievals from the higher frequency observations (i.e., X- and Ku-bands) show adverse influence by a much thinner vegetation cover. Soil Moisture retrievals with a soil moisture error estimate beyond 0.2 m³m^-3 are considered to be unreliable and are masked out

Figure 4: Error of soil moisture as related to the vegetation optical depth for 3 different frequency bands (from Parinussa et al., 2011).

For the new L-band based retrievals from SMOS, the vegetation influence is less as compared to the C-, X- and Ku-band retrievals, which can be seen from the R_value and Triple Collocation Analysis (TCA) results in Figure 5. In Figure 5, the SMOS LPRM and AMSR-E LPRM (based on C-band) are included and shows more stable results over dense vegetation, i.e. NDVI values of over 0.45. A complete analysis of the error for L-band soil moisture, comparable to the results from Figure 3, are planned in the near future.

Figure 5: triple collocation analysis (TCA: top) and Rvalue results (bottom) for several soil moisture datasets, including SMOS LPRM and AMSR-E LPRM, for changing vegetation density (NDVI). Based on Van der Schalie et al. (2018).

3.1.4.2.  Frozen surfaces and snow

Under frozen surface conditions the dielectric properties of the water changes dramatically and therefore all pixels where the surface temperature is observed to be at or below 273 K are assigned with an appropriate data flag, this was determined using the method of Holmes et al. (2009).

3.1.4.3. Water bodies

Water bodies within the satellite footprint can strongly affect the observed brightness temperature due to the high dielectric properties of water. Especially when the size of a water body changes over time they can dominate the signal. LPRM uses a 5 % water body threshold based on MODIS observations and pixels with more than 5 % surface water are masked.

3.1.4.4. Rainfall

Rainstorms during the satellite overpass affect the brightness temperature observation, and are therefore flagged in LPRM. The flagging system for active rain is based on the rainfall index of Seto et al. (2005). This method makes use of the vertical polarized 36.5 GHz and 19 GHz observations to detect a rain event. Index values of 5 and beyond are used to identify an active rainstorm. Soil moisture retrievals with these index values are flagged.

3.1.4.5. Radio Frequency interference

Natural emission in several low frequency bands are affected by artificial sources, so called Radio Frequency Interference (RFI). As a diagnostic for possible errors an RFI index is calculated according to De Nijs et al. (2015). Most passive microwave sensors that are used for soil moisture retrieval observe in several frequencies. This allows LPRM to switch to higher frequencies in areas affected by RFI. The new methodology that is now used for RFI detection uses the estimation of the standard error between two different frequencies. It uses both the correlation coefficient between two observations and the individual standard deviation to determine the standard error in Kelvin. A threshold value of 3 Kelvin is used to detect RFI. This method does not produce false positives in extreme environments and is more sensitive to weak RFI signals in relation to the traditional methods (e.g. Li et al., 2004).

As the currently integrated SMOS mission does not have multiple frequencies to apply this method, here we base the filtering on the RFI probability information that is supplied by in the SMOS Level 3 data.

3.2. Active Microwave Algorithm

The active microwave soil moisture products utilized in the generation of the C3S soil moisture datasets are obtained from external operational sources as follows:

• ERS-1 AMI surface soil moisture products have been generated at TU Wien (2013) utilizing change detection method described in Wagner et al. 1999b.
• ERS-2 AMI surface soil moisture data sets stem from reprocessing activities which have been carried out within ESA's SCIRoCCo project (Crapolicchio et al., 2016). The ERS-2 data set used in all ESA CCI SM versions is the ERS.SSM.H.TS 25 km soil moisture time series product (ESA, 2017).
• Metop ASCAT surface soil moisture data sets stem from the EUMETSAT Satellite Application Facility on Support to Operational Hydrology and Water Management (H SAF, http://h-saf.eumetsat.int/). ESA CCI SM v04.7 uses both the H SAF H113 Metop ASCAT SSM CDR2017 (H SAF, 2018a) and the H SAF H114 Metop ASCAT SSM CDR2017-EXT (H SAF, 2018b).

A detailed overview of the retrieval methodology employed in the generation of the Surface Soil Moisture products from the ERS-1 AMI instrument is provided in Wagner et al. 1999b, for the Surface Soil Moisture products from the ERS-2 AMI instrument please refer to Crapolicchio et al. 2004, and for the ASCAT derived products an overview of the retrieval methodology can be found in H-SAF 2018c.

3.3. Merging Strategy

This chapter is based upon the “Algorithm Theoretical Baseline Document (ATBD) D2.1 Version 04.5” [D1] document.

3.3.1. Principle of the merging process

The generation of the long-term soil moisture data set involves three steps (Figure 6): (1) merging the original passive microwave soil moisture products into one product, (2) merging the original active microwave soil moisture products into one product, and (3) merging all original active and passive microwave soil moisture products into one. The input datasets considered for the generating and validating the merged soil moisture product are:

• Scatterometer-based soil moisture products
  • Sensors: ERS-1/2 and Metop-A ASCAT, Metop-B ASCAT
  • Retrieval method: TU Wien change detection method (Wagner et al. 1999b; H-SAF 2018c)
  • Time span: 1991 – now
• Radiometer-based soil moisture products
  • Sensors: SMMR, SSM/I, TRMM, AMSR-E, AMSR2, WindSat, and SMOS
  • Retrieval method: VUA-NASA LPRM v5/v6 model inversion packages (Owe et al. 2008; van der Schalie et al. 2015)
  • Time span: 1978 – now
• Modelled 0 – 10 cm soil moisture from the Noah land surface model of the Global Land Data Assimilation System version 2.0 (GLDAS; Rodell et al., 2004).
  • Time span: 1948 – 2010 (0.25 degree resolution)

The homogenized and merged product presents surface soil moisture with a global coverage and a spatial resolution of 0.25°. The time period spans the entire period covered by the individual sensors, i.e. 1978 – now, while measurements are provided at a 1-day sampling.

 
Figure 6: Overview of the three-step blending approach from original products to the final blended active & passive microwave soil moisture product (Adapted from Liu et al. 2012)

3.3.2. Overview of processing steps

The level 2 surface soil moisture products derived from the active and passive remotely sensed data undergo a number of processing steps in the merging procedure (see Figure 7 for an overview):

1. Temporal Resampling
2. Spatial Resampling
3. Rescaling passive and active level 2 observations into radiometer and scatterometer climatologies (for the ACTIVE and PASSIVE product), and separately rescaling all level 2 observations into a common climatology (for the COMBINED product)
4. Triple collocation analysis (TCA) based error characterisation of all rescaled level 2 products
5. Polynomial regression between VOD and error estimates
6. Derivation of error estimates from the VOD regression in regions where they were not available after (4), i.e., where TCA is deemed unreliable
7. Merging rescaled passive and active time series into the PASSIVE, ACTIVE, and COMBINED product, respectively

Figure 7: Overview of the processing steps in the C3S product generation: The merging  of two or more data sets is done by weighted averaging and involves overlapping time periods, whereas the process of joining data sets only concatenates two or more data sets between the predefined time periods. The join process is performed on datasets of each lines and on datasets separated by comma within the rectangular process symbol. *The [SSM/I, TMI] period is specified not only by the temporal, but also by the spatial latitudinal coverage.

3.3.3. Methodology

In this section the algorithms of the scaling and merging approach are described. Notice that several algorithms, e.g. rescaling, are used in various steps of the process, but will be described only once

3.3.3.1. Resampling

The sensors used for the different merged products have different technical specifications (Table 2). Obvious are the differences in spatial resolution and crossing times. Both elements need to be brought into a common reference before the actual merging can take place.

3.3.3.2. Spatial Resampling

The merged products are provided on a regular grid with a spatial resolution of 0.25° in both latitude and longitude extension. This is a trade-off between the higher resolution scatterometer data and the generally coarser passive microwave observations without leading to any undersampling. The resolution of the products is often adopted by land surface models. Nearest neighbour resampling is performed on the radiometer input data sets to bring them into the common regular grid. Following this resampling technique each grid point in the reference (regular grid) data set is assigned to the value of the closest grid point in the input dataset. In general, the nearest neighbour resampling algorithm can be applied to data set with regular degree grid. For the active microwave data sets, where equidistant grid points are defined by the geo-reference location of the observation, the hamming window function is used to resample the input data to a 0.25° regular grid. The search radius is a function of latitude of the observation location, as the distance between two regular grid points reduces as the location tends towards the poles. In contrast, the active microwave data set uses the DGG, where the distance between every two points is the same. This main difference between the DGG (active) and the targeted regular degree grid is rectified by using a hamming window with search radius dependent on the latitude for the spatial resampling of the active microwave data.

Table 3: Major characteristics of passive and active microwave instruments and model product

*For passive and active microwave instruments, this stands for the footprint spatial resolution.

3.3.3.3. Temporal Resampling

The temporal sampling of the merged product is 1 day. The reference time for the merged dataset is set at 0:00 UTC. For each day starting from the time frame centre at 0:00 UTC observations within ±12 hours are considered. The elaborated temporal resampling strategy firstly searches for the valid observation that is closest to the reference time. In case there are only invalid observations, which are flagged other than "0" (zero), within a certain time frame, the closest measurement among these invalid observations is selected. In the event that there are no measurements available at all within a time frame, no action is taken. This strategy results in data gaps when no observations within ±12 hours from the reference time are available. For the modelled soil moisture datasets, no resampling is required as they already include the reference time stamp of 0:00 UTC. The LPRM (passive) soil moisture estimates based on night-time (often the descending mode) observations are more reliable than those obtained during the day (often the ascending mode). This is mainly caused by the complexity to derive accurate estimates of the effective surface temperature during the day. For this reason, only night-time soil moisture observations from radiometers are used for the merged product.

3.3.3.4. Rescaling

Due to different observation frequencies, observation principles, and retrieval techniques, the contributing soil moisture datasets are available in different observation spaces. Therefore, before merging can take place at either level, the datasets need to be rescaled into a common climatology. The rescaling procedure is applied to the daily soil moisture values at three levels in the processing chain:

1. Rescaling of all the passive microwave soil moisture observations to the climatology of AMSR-E.
2. Rescaling of all the active microwave soil moisture observations to the climatology of ASCAT
3. (Separately) Rescaling of all the active and passive microwave datasets to the climatology of GLDAS-Noah v2.1 for the COMBINED product only. Where there is an overlap between GLDAS-Noah v2.1 and the sensor, this period is used; where there is no overlap (for ERS, SMMR, SSMI and TMI) the whole period is used).

Scaling is performed using cumulative distribution function (CDF) matching which is a well-established method for calibrating datasets with deviating climatologies (Drusch et al. 2005; Liu et al. 2007; Liu et al. 2011; Reichle et al. 2004). CDF-matching is applied for each grid point individually and based on piece-wise linear matching. This variation of the CDF-matching technique proved to be robust also for shorter time periods (Liu et al. 2011). The matching is shown by means of an example for a grid point centred at 41.375^oN, 5.375^oW. Figure 8 shows for this location the time series of soil moisture estimates from GLDAS-Noah, AMSR-E and ASCAT, respectively. CDF-matching for this time series is performed in the following way:

1. Time steps (0:00 UTC ±12h) are identified for which all data sets provide valid soil moisture values.
2. For the time-collocated data points CDFs are computed (Figure 9 a-c).
3. For each CDF curve the 0, 5, 10, 20, 30, 40, 50, 60, 70, 80, 90, 95 and 100 percentiles are identified.
4. Use the 13 percentiles of the CDF curves to define 12 segments. The CDF curves of these circled values are shown in Figure 9a, b and c.

5. The 13 percentile values from the AMSR-E and ASCAT CDF curves are plotted against those of Noah (Figure 9d and e) and scaling linear equations (e.g., slope and intercept) between two consecutive percentiles are computed.

   $$slope_i = \frac{pref_{i+1} - pref_i}{psrc_{i+1} - psrc_i} \quad Eqn. 39$$ $$intercept_i = pref_i - (psrc_i \ast slope_i) \quad Eqn. 40$$

   where i=1..12, is the number of the segments, and pref is the percentile of the GLDAS-Noah data (reference), and psrc is the percentile of either AMSR-E or ASCAT data (source) respectively.

6. The obtained linear equations are used to scale all observations of the target data set (i.e., also the time steps that do not have a corresponding observation in the reference data set) to the climatology of the reference data set (Figure 9f).

   $$sm_r = slope_i \ast sm + intercept_i \quad Eqn. 41$$

   where smr is the rescaled soil moisture and sm is the original soil moisture value. slopei and intercepti are chosen depending on the sm value and its corresponding i-percentile. The AMSR-E and ASCAT values outside of the range of CDF curves can also be properly rescaled, using the linear equation of the closest value.

Figure 8: Time series of soil moisture estimates from (a) Noah, (b) AMSR-E and (c) ASCAT for a grid cell (centered at 41.375° N, 5.375° W) in 2007. Circles represent days when Noah, AMSR-E and ASCAT all have valid estimates. (Figure taken from Liu et al. 2011)

Figure 9: Example illustrating how the cumulative distribution function (CDF) matching approach was implemented to rescale original AMSR-E and ASCAT against Noah soil moisture product in this study. (a, b, c) CDF curves of AMSR-E, Noah and ASCAT soil moisture estimates for the grid cell shown in Fig. 2. (d) Linear regression lines of AMSR-E against Noah for 12 segments. (e) Same as (d), but for ASCAT and Noah. (f) CDF curves of Noah (black), rescaled AMSR-E (blue) and rescaled ASCAT (red) soil moisture products. (Figure taken from Liu et al. 2011)

3.3.3.5. Error characterization

Errors in the individual active and passive products are characterized by means of triple collocation analysis. These errors are used both for estimating the merging parameters and for characterizing the errors of the merged product (see section 3.3.4).

3.3.3.6. Triple collocation analysis

Triple collocation analysis is a statistical tool that allows estimating the individual random error variances of three data sets without assuming any of them acting as supposedly accurate reference (Gruber et al. 2016a & 2016b). This method requires the errors of the three data sets to be uncorrelated, therefore triplets always comprise of i) an active data set, (ii) a passive data set, and (iii) the GLDAS-Noah land surface model, which are commonly assumed to fulfil this requirement (Dorigo et al. 2010). Error variance estimates are obtained as:

where σ_ε² denotes the error variance; σ²and σ denote the variances and covariances of the data sets; and the superscripts denote the active (a), the passive (p), and the modelled (m) data sets, respectively. For a detailed derivation see Gruber et al. (2016b). Notice that these error estimates represent the average random error variance of the entire considered time period, which is commonly assumed to be stationary. Furthermore, the soil moisture uncertainties of the three products (ACTIVE, PASSIVE, and COMBINED) are determined by the above equations.

3.3.3.7. Error gap-filling

TCA does not provide reliable error estimates in all regions, mainly if there is no significant correlation between all members of the triplet, which often happens for example in high-latitude areas or in desert areas. TCA error estimates are therefore disregarded in case of insignificant Pearson correlation (p-value < 0.05) between any of the data sets. In these areas, error estimates are derived from the mean VOD (derived from AMSR-E in the entire mission period) at that particular location:

Where the subscript denotes the spatial location; and the parameters ai are derived from a global polynomial regression between VOD and TCA based error estimates at locations where they are considered reliable (i.e., all data sets are significantly correlated). For TMI and WINDSAT third order polynoms (N=3) are used and for all other sensors second order polynoms (N=2) are used, which was empirically found to provide the best regression results.

3.3.3.8. Merging

The merging procedure consists of (1) merging the original passive microwave product into the PASSIVE product, (2) merging the original active microwave products into the ACTIVE product, and (3) merging the original active and passive microwave products into the COMBINED product. The merging is performed by means of a weighted average which takes into account the error properties of the individual data sets that are being merged. Such weighted average is calculated as

where Θm denotes the merged soil moisture product; Θi are the soil moisture products that are being merged, and wi are the merging weights.

3.3.3.9. Weight estimation

Per definition, the optimal weights for a weighted average are determined by the error variances of the input data sets and write as follows:

where the superscripts denote the respective data sets; i is the data set for which the weight is being calculated; and N is the total number of data sets which are being averaged. The required error variances are calculated using Eqn. 42. Notice that error covariance terms are neglected as they cannot be estimated reliably.

It should be mentioned that the above definition of the weights based on error variances assumes all data sets to be in the same data space. However, data sets usually vary in their signal variability due to algorithmic differences, varying signal frequencies, etc. Therefore, conceptually, it is more appropriate to define relative weights in terms of the data sets Signal-to-Noise Ratio (SNR) properties rather than of their error variance (Gruber et al., 2017). Nevertheless, the actual merging requires a harmonization of the data sets into a common data space, which in the case of the CCI SM data set is done using the CDF matching approach described in Section 3.3.3.4 . Therefore, the calculation of the weights using Eqn. 44 suffices, keeping in mind that they represent the rescaled error variances of rescaled data sets.

3.3.3.10. Merging passive microwave products

Differences in sensor specifications, particularly in microwave frequency and spatial resolution, result in different absolute soil moisture values from SMMR, SSM/I, TMI and AMSR-E. Even though SMMR and AMSR-E have a similar frequency (i.e., C-band), their absolute values are different. Therefore, a Spearman and Pearson correlation analysis was performed between the different soil moisture products to identify differences and correspondences between the data sets (Liu et al. 2012). Based on this analysis, the AMSR-E soil moisture retrievals were identified as more accurate than the other passive products due to the relatively low microwave frequency and high temporal and spatial resolution of the sensor. Thus, soil moisture retrievals from AMSR-E are selected as the reference to which soil moisture retrievals from SMMR, SSM/I, TMI, WindSat, and SMOS are rescaled and merged on a pixel basis according to the following steps. AMSR2 data that is used for generating the product were not rescaled as there is no overlapping time period between AMSR-E and AMSR2.
Merging SSM/I and TMI with AMSR-E

1. Rescale original TMI against the AMSR-E reference using the piece-wise linear cumulative distribution function (CDF) matching technique (Section 3.3.3.4) based on their overlapping period (Figure 10a)

2. Decompose SSM/I and AMSR-E time series into their own seasonality and anomalies (Figure 9b). This is done for their overlapping period from July 2002 through December 2007. The seasonality for each sensor was calculated by taking the average of the same day of the year for their overlapping period. The seasonality ( ) is one time series of 366 values, one value for each day of the year (DOY):

   $$\overline{SM}_{DOY} = \left( \sum_{YR=2002}^{2007} SM_{DOY}^{YR} \right) /N \quad Eqn. 45 $$

   where YR represents the year 2002 through 2007; N represents the number of valid soil moisture retrievals. The value of is only taken from the year 2004 as that is the only leap year (i.e., 366 days) between 2002 and 2007. The anomalies (ANO) over their individual entire periods were obtained by removing the sensor's seasonality from the original (ORI) time series:

   $$ ANO_{DOY}^{YR} = ORI_{DOY}^{YR} - \overline{SM}_{DOY} \quad Eqn. 46 $$

   where YR represents the year 1987 through 2007 for SSM/I and 2002 through October 2011 for AMSR-E.

3. Rescale "anomalies of SSM/I" against "anomalies of AMSR-E" using the piece-wise linear CDF matching technique (Figure 10c).
4. Add the AMSR-E seasonality to the "rescaled SSM/I anomalies" (from Step 3) and obtain reconstructed SSM/I (Figure 10d).
5. Merge the reconstructed SSM/I, rescaled TMI, and original AMSR-E to obtain the merged SSM/I-TMI-AMSR-E dataset Figure 10e). The lower the measurement frequency, the more accurate soil moisture retrievals can be expected. Therefore AMSR-E is used for July 2002 – December 2008 and the rescaled TMI is used for January 1998 – June 2002 between N40o and S40o. Otherwise the reconstructed SSM/I is used.

Merging SMMR with SSM/I-TMI-AMSR-E
The overlapping period between SMMR and other sensors is too short to perform the rescaling as conducted on retrievals from other sensors. In order to incorporate SMMR (1979 – 1987) soil moisture retrievals into the merged product, we assumed that the dynamic range of SMMR retrievals is the same as the range of merged SSM/I-TMI-AMSR-E dataset. Following this assumption, we produced the rescaled SMMR (Nov 1978 to July 1987) by matching the CDF curve of SMMR against that of the merged SSM/I–TMI–AMSR-E dataset for each grid point. The CDF curve is calculated based on all observation of both data sets. Together with the merged SSM/I-TMI-AMSR-E dataset, we obtained the merged SMMR-SSM/I-TMI-AMSR-E soil moisture product covering the period Nov 1978 – Sep 2007 (Figure 10). It should be emphasized that the CDF matching process changes the absolute values of SMMR, SSM/I and TMI products, but does not change the relative dynamics of the original retrievals, which is demonstrated in Liu et al. (2011).

Table 4: Used passive sensors in the PASSIVE product

Merging SMOS, WindSat, and AMSR2 with SMMR-SSM/I-TMI-AMSR-E
WindSat data (1 October 2007 to 31 June 2012) bridge the operational time gap between AMSR-E, which failed to deliver data since 4 October 2011, and AMSR2, for which data are available from 02 July 2012 onward. SMOS data in ascending satellite mode are available from 1 July 2010 onwards. The CDFs between WindSat and SMOS on the one hand and AMSR-E on the other are calculated based on their respective overlapping time periods with AMSR-E. Within the time period from 1 October 2007 to May 2015 there are various combinations of data overlap. Figure 1, Figure 10, Table 3, and Table 4 illustrate these overlaps. The data periods AMSR-E & WindSat (1 October 2007 to 30 June 2010), AMSR-E & WindSat & SMOS (1 July 2010 to 3 October 2011), WindSat & SMOS (4 October 2011 to 30 June 2012), and AMSR2 & SMOS (1 July 2012 to NRT). AMSR2 was previously scaled to the rescaled WindSat data as described in Parinussa et al. (2015) because there is no overlap with AMSR-E data. The resulting product hereafter is referred to as the PASSIVE product. The following paragraph describes in more detail the process of merging these datasets, when more than one sensor is used.

Figure 10: Example illustrating how (a) the TMI was rescaled against AMSR-E, (b-e) the SSM/I anomalies were rescaled against AMSRE-E anomalies, reconstructed and merged with rescaled TMI and AMSR-E, and (e) the SMMR was rescaled and merged with the others. The grid cell is centred at 13.875°N, 5.875°W.). (Image courtesy Liu et al. 2012)

Merging in periods where more than one sensors is used
As it can be seen from Figure 1 there are four distinct periods where more than one passive dataset is available, i.e., AMSR-E & WindSat, AMSR-E & WindSat & SMOS, WindSat & SMOS, and SMOS & AMSR-2. In these periods, a weighted average of the respective sensors is used to construct the merged PASSIVE product. Error estimates are obtained from triple collocation analysis (see 3.3.3.6) using ASCAT and GLDAS-Noah data to complement the respective triplets. Notice that for certain locations triple collocation analysis does not yield valid error estimates e.g., due to numerical issues. In such cases, weights are equally distributed amongst the available sensors (e.g. 0.33 for AMSR-E, WindSat, and SMOS if all three datasets are available). Also, soil moisture estimates of the various sensors are not available every day, hence there are certain dates during the overlapping periods on which not all data sets provide a valid estimate to calculate the weighted average. In such cases, the weights are re-distributed amongst the remaining data sets, again based on their relative SNR properties. However, this re-distribution of weights could significantly worsen data quality on these days because of the increasing contribution of measurements which initially would have had a low weight due to their (relatively) low SNR. Therefore, soil moisture estimates in the merged product on days where not all data sets provide valid estimates are set to NaN, if the sum of the initial weight of the remaining data sets is lower than 1/(2N) where N is the total number of data sets that are potentially available for the corresponding merging period. This threshold has been derived empirically to provide a good trade-off between temporal measurement density and average data quality.

3.3.3.11. Merging active microwave products

Different sensor specifications between ERS1/2 and ERS2 (e.g. spatial resolution) need to be compensated by using the same rescaling techniques performed on the radiometer data sets. The CDF curves for ERS2 are calculated based on the overlap with ERS1/2. Rescaling ERS2 against ERS1/2 and then merging them generates the AMI-WS active data set, which is subsequently scaled and merged to the MetOp-A ASCAT data (Figure 6). Table 4 and Figure 13a show the sensors used in the ACTIVE product for the individual time periods.

Table 5: Used active sensors in the ACTIVE product

An example of a soil moisture time series from AMI-WS ERS1/2 and MetOp-A ASCAT for the grid point centred at 13.875°N, 5.875°W (Niger River basin in southern Mali) is shown in Figure 11, where the AMI-WS ERS1/2 is labelled as SCAT to denote its predecessor role to ASCAT. The AMI-WS ERS1/2 and MetOp-A ASCAT soil moisture variations are scaled between the lowest (0%) and highest (100%) values over their individual operational period. The limited overlap in time (i.e., a few months) and space (i.e. only Europe, Northern America and Northern Africa) rules out the global adjustment method based on the information of their overlapping period, such as applied between TMI and AMSR-E. Figure 11 also shows the evident AMI-WS ERS1/2 data gap from 2001 to 2003.

As retrievals from MetOp-A ASCAT and AMI-WS capture similar seasonal cycles (Liu et al. 2011), we assume that their dynamic ranges are identical and use for each grid point the CDF curves of both datasets to rescale AMI-WS to MetOp-A ASCAT before merging them (Figure 11b). MetOp-A ASCAT data from 1 January 2007 to 5 November 2012 are joined with AMI-WS data from 5 August 1991 to 31 December 2006. In the time period from 6 November 2012 to 31 December 2015 Metop-A ASCAT and MetOp-B ASCAT data are available. These two datasets are merged by applying the arithmetic average for locations, where both observations are available, otherwise either one of the two is then used. Joining AMI-WS & Metop-A ASCAT from 5 August 1991 to 20 July 2015 with Metop-A ASCAT&Metop-B ASCAT from 21 July 2015 onwards generates the ACTIVE product (Figure 11).

 
Figure 11: Example illustrating fusion of ERS1/2 (SCAT) with ASCAT. Note the data gap from 2001 – 2003, which will be filled by ERS2 data as shown in Figure 7. The grid point is centered at 13.875°N, 5.875°W.). (Image courtesy Liu et al. 2012)

Figure 12: Rescaling the merged passive and active microwave product against the GLDAS-1-Noah simulation. (a) GLDAS-1-Noah soil moisture; (b) merged passive microwave product and one rescaled against GLDAS-1-Noah; (c) same as (b) but for active microwave product. The grid cell is centred at 13,875°N, 5.875°W.). (Image courtesy Liu et al. 2012)

3.3.4. Merging passive and active microwave products

For generating the combined product, climatologies of all passive and active level 2 data sets are first harmonized by rescaling against GLDAS-2.

Table 6: Used sensors in individual time periods.

Similar to the generation of the PASSIVE product, relative weights at each time step are derived from the TCA- or VOD-regression based error estimates for each individual sensor. Depending on how many sensors are available within a particular period, a (1/2N) threshold for the minimum weight of a particular sensor was applied if not all sensors provide a soil moisture estimate at that day.

3.3.5. Near real time (NRT) mode of the processor

The processing steps described in the previous sections work on the complete time series of the datasets. This is not feasible during NRT production of the products. Figure 13 gives an overview of the NRT processing chain. The parameters for CDF scaling and the characterised errors are precomputed using data until 30/06/2017 and used directly in the ICDR processing. This speeds up the processing and results in a consistent time series based on stable merging parameters.

Figure 13: Overview of NRT processing chain

4. Output data

Table 7 lists the output fields generated by the processing system and available to data users. Detailed information about each field and the file format are provided in the Product User Guide and Specification (PUGS) [D2].

Table 7: Overview of output data fields

References

Ashcroft, P. and Wentz, F. (2000). Algorithm Theoretical Basis Document: AMSR Level-2A Algorithm, Revised 03 November. Santa Rosa, California USA: Remote Sensing Systems.

Attema, E., & Ulaby, F. (1978). Vegetation modeled as water cloud. Radio Science, 13, 357-364

Balsamo, G., Albergel, C., Beljaars, A., Boussetta, S., Brun, E., Cloke, H., Dee, D., Dutra, E., Muñoz-Sabater, J., Pappenberger, F., de Rosnay, P., Stockdale, T., and Vitart, F. (2015), “ERA-Interim/Land: a global land surface reanalysis dataset”, Hydrol. Earth Syst. Sci., vol. 19, pp. 389-407, doi: 10.5194/hess-19-389-2015.

Balsamo, G., S. Boussetta, P. Lopez, L. Ferranti (2010), Evaluation of ERA-Interim and ERA-Interim-GPCP-rescaled precipitation over the U.S.A., ERA Report Series, n. 5, pp10 [pdf].

Bartalis, Z., Wagner, W., Naeimi, V., Hasenauer, S., Scipal, K., Bonekamp, H., Figa, J., & Anderson, C. (2007). Initial soil moisture retrievals from the METOP-A Advanced Scatterometer (ASCAT). Geophy. Res. Lett, 34, L20401

Beaudoing, Hiroko and M. Rodell, NASA/GSFC/HSL (2016), GLDAS Noah Land Surface Model L4 3 hourly 0.25 x 0.25 degree V2.1, Greenbelt, Maryland, USA, Goddard Earth Sciences Data and Information Services Center (GES DISC), Accessed: [Data Access Date], 10.5067/E7TYRXPJKWOQ

Bennartz, R., and Michelson, D.B. (2012), Validation of AMSR-E and AMSU/HSB level 1 brightness temperatures and level 2 cloud and precipitation parameters, NAG5--12579, Meteorological and Hydrological Institute, Report, March 2004. 

Bevington, P.R., & Robinson, D.K. (2002). Data reduction and error analysis for the physical sciences. (3th ed.). Boston: McGraw-Hill Science/Engineering/Math

Bevis, M., Businger, S., Herring, T., Rocken, C., Anthes, R., & Ware, R. (1992). GPS Meteorology: Remote Sensing of Atmospheric Water Vapor Using the Global Positioning System. Journal of Geophysical Research, 97, null-15801

CATDS SMOS L3 soil moisture retrieval processor Algorithm Theoretical Baseline Document (ATBD) (2013),
http://www.cesbio.ups-tlse.fr/SMOS_blog/wp-content/uploads/2013/08/ATBD_L3_rev2_draft.pdf

CATDS LEVEL 3 DATA PRODUCT DESCRIPTION Soil Moisture and Brightness Temperature (2014), http://www.cesbio.ups-tlse.fr/SMOS_blog/wp-content/uploads/DOCS/SO-TN-CB-CA-0001.3a.pdf

Crapolicchio, R., A. Bigazzi, G. De Chiara, X. Neyt, A. Stoffelen, M. Belmonte, W. Wagner, C. Reimer (2016) The scatterometer instrument competence centre (SCIRoCCo): Project's activities and first achievements, Proceedings European Space Agency Living Planet Symposium 2016, 9-13 May 2016, Prague, Czech Republic, 9-13.

Crow, W.T., Wagner, W., & Naeimi, V. (2010). The impact of radar incidence angle on soil moisture retrieval skill. IEEE Geoscience and Remote Sensing Letters, 7, 501-505

De Jeu R.A.M, T.R.H. Holmes, R. M. Parinussa, M Owe (2014), A spatially coherent global soil moisture product with improved temporal resolution, Journal of Hydrology 516, 284-296.

De Nijs, A.H., Parinussa, R.M., De Jeu, R.A.M., Schellekens, J. and Holmes, T.R. (2015), “A methodology to determine radio-frequency interference in AMSR2 observations”, IEEE Transactions on Geoscience and Remote Sensing, vol. 53, no. 9, pp. 5148-5159.

Dee, D.P. et al., 2011. The ERA-Interim reanalysis: configuration and performance of the data assimilation system. Quarterly Journal of the Royal Meteorological Society, 137(656), pp.553–597

Dobson, M.C., Ulaby, F.T., Hallikainen, M.T., & Elrayes, M.A. (1985). Microwave dielectric behavior of wet soil. Part II: Dielectric mixing models. IEEE Transactions on Geoscience and Remote Sensing, 23, 35-46

Dorigo, W.A., et al. (2011) "The International Soil Moisture Network: A data hosting facility for global in situ soil moisture measurements", Hydrology and Earth System Sciences 15 (5), pp. 1675-1698

Dorigo, W.A., Scipal, K., Parinussa, R.M., Liu, Y.Y., Wagner, W., De Jeu, R.A.M., and Naeimi, V. (2010), “Error characterisation of global active and passive microwave soil moisture datasets”, Hydrology and Earth System Sciences, vol. 14, pp. 2605 – 2616, doi: 10.5194/hess-14-2605-2010.

Dorigo, W., Wagner, W., Albergel, C., Albrecht, F., Balsamo, G., Brocca, L., Chung, D., Ertl, M., Forkel, M., Gruber, A., et al. (2017), “ESA CCI Soil Moisture for improved Earth system understanding: state-of-the art and future directions”, Remote Sensing of Environment, https://doi.org/10.1016/j.rse.2017.07.001

Drusch, M., Wood, E., & Gao, H. (2005). Observation operators for the direct assimilation of TRMM microwave imager retrieved soil moisture. Geophysical Research Letters, 32, L15403

ESA (2017) ERS-2 SCATTEROMETER Surface Soil Moisture Time Series in High Resolution - ERS.SSM.H.TS (25 km Time-Series product), SCI-MAN-16-0047-v02 https://earth.esa.int/documents/700255/3799027/scirocco-pum-ts.pdf/cb893b39-b7db-441a-a821-9328436aa8f5

FAO/IIASA/ISRIC/ISS-CAS/JRC, 2009. Harmonized World Soil Database (version 1.1). FAO, Rome, Italy and IIASA, Laxenburg, Austria

Fu, C. C., D. Han, S. T. Kim, and P. Gloersen (1988). User's guide for the Nimbus- 7 Scanning Multichannel Microwave Radiometer (SMMR) CELL-ALL tape. NASA Reference Publication #1210, National Aeronautics and Space Administration, Washington, D.C.        

Fung, A.K. (1994). Microwave scattering and emission models and their applications. Boston: Artech House

Gaiser, P.W., K. M. St. Germain, E. M. Twarog, G. A. Poe, W. Purdy, D. Richardson, W. Grossman, W. L. Jones, D. Spencer, G. Golba, J. Cleveland, L. Choy, R. M. Bevilacqua and P. S. Chang (2004), "The WindSat spaceborne polarimetric microwave radiometer: Sensor description and early orbit performance," IEEE Trans. Geosci. Remote Sens., vol. 42, 2347-2360.

Gloersen, P., D. J. Cavalieri, A. T. C. Chang, T. T. Wilheit, W. J. Campbell, O. M. Johannessen, K. B. Katsaros, K. F. Kunzi, D. B. Ross, D. Staelin, E. P. L. Windsor, F. T. Barath, P. Gudmansen, E. Langham, and R. Ramseier (1984). A summary of results from the first Nimbus-7 SMMR observations. J. Geophys. Res. 89, 5335-5344.

Gruber, A., Dorigo, W., Crow, W., and Wagner, W. (2017), “Triple Collocation-Based Merging of Satellite Soil Moisture Retrievals”, IEEE trans. geosc. rem. sens., DOI: 10.1109/TGRS.2017.2734070

Gruber, A., Su, C.H., Crow, W.T., Zwieback, S., Dorigo, W.A., & Wagner, W. (2016a). Estimating error cross-correlations in soil moisture data sets using extended collocation analysis. Journal of Geophysical Research: Atmospheres, 121(3), 1208-1219.

Gruber, A., Su, C.H., Zwieback, S., Crow, W.T., Wagner, W., & Dorigo, W. (2016b). Recent advances in (soil moisture) triple collocation analysis. International Journal of Applied Earth Observation and Geoinformation, 45, 200-211.

H SAF (2018a). ASCAT Surface Soil Moisture CDR2017 time series 12.5 km sampling – Metop (H113), EUMETSAT SAF on Support to Operational Hydrology and Water Management. http://dx.doi.org/10.15770/EUM_SAF_H_0005

H SAF (2018b). ASCAT Surface Soil Moisture CDR2017-EXT time series 12.5 km sampling – Metop (H114), EUMETSAT SAF on Support to Operational Hydrology and Water Management. 

H SAF (2018c) Algorithm Theoretical Baseline Document (ATBD), Metop ASCAT Soil Moisture Data Records v0.7 (H113)

Hillburn KA, and CL Shie, (2011) Decadal trends and variability in special sensor microwave imager (SSM/I) Brightness Temperatures and Earth Incidencs angle, NASA RSS Technical Report 092811 

Holmes, T.R.H., De Jeu, R.A.M., Owe, M., & Dolman, A.J. (2009). Land surface temperature from Ka band (37 GHz) passive microwave observations. Journal of Geophysical Research -Atmospheres, 114

Hsieh, C.-Y., Fung, A.K., Nesti, G., Sieber, A.J., & Coppo, P. (1997). A further study of the IEM surface scattering model. IEEE Transaction on Geoscience and Remote Sensing, 35, 901-909

Kerr, Y.H., Waldteufel, P., Wigneron, J.P., Delwart, S., Cabot, F., Boutin, J., Escorihuela, M.J., Font, J., Reul, N., Gruhier, C., Juglea, S.E., Drinkwater, M.R., Hahne, A., Martin-Neira, M., and Mecklenburg, S. (2010), “The SMOS mission: New tool for monitoring key elements of the global water cycle”, Proceedings of the IEEE, vol. 98, no. 5, doi: 10.1109/JPROC.2010.2043043.

Kottek, M., Grieser, J., Beck, C., Rudolf, B., & Rubel, F. (2006). World Map of the Köppen-Geiger climate classification updated. Meteorologische Zeitschrift, 15, 259-263

Lehner, B. & Döll, P., 2004. Development and validation of a global database of lakes, reservoirs and wetlands. Journal of Hydrology, 296(1-4), pp.1–22

Li, L., Njoku, E.G., Im, E., Chang, P.S., and St. German, K. (2004), “A preliminary survey of radio-frequency interference over the U.S. in Aqua AMSR-E data”, IEEE Trans. Geosc., vol. 42, no. 2, pp. 380 – 390, doi: 10.1109/TGRS.2003.817195.

Liu, Y., de Jeu, R.A.M., van Dijk, A.I.J.M., & Owe, M. (2007). TRMM-TMI satellite observed soil moisture and vegetation density (1998-2005) show strong connection with El Nino in eastern Australia. Geophysical Research Letters, 34, Art. No. L15401

Liu, Y.Y., Dorigo, W.A., Parinussa, R.M., De Jeu, R.A.M., Wagner, W., McCabe, M.F., Evans, J.P., & Van Dijk, A.I.J.M. (2012). Trend-preserving blending of passive and active microwave soil moisture retrievals. Remote Sensing of Environment, 123, 280-297

Liu, Y.Y., Parinussa, R.M., Dorigo, W.A., de Jeu, R.A.M., Wagner, W., van Dijk, A., McCabe, F.M., & Evans, J.P. (2011). Developing an improved soil moisture dataset by blending passive and active microwave satellite-based retrievals. Hydrology and Earth System Sciences, 15, 425-436

Meesters, A., De Jeu, R.A.M., & Owe, M. (2005). Analytical derivation of the vegetation optical depth from the microwave polarization difference index. IEEE Geoscience and Remote Sensing Letters, 2, 121-123

Mo, T., Choudhury, B.J., Schmugge, T.J., Wang, J.R., & Jackson, T.J. (1982). A model for microwave emission from vegetation-covered fields. Journal of Geophysical Research-Oceans and Atmospheres, 87, 1229-1237

Mironov, V.L., Dobson, M.C., Kaupp, V.H., Komarov, S.A., & Kleshchenko, V.N. (2004). Generalized refractive mixing dielectric model for moist soils. IEEE Transactions on Geoscience and Remote Sensing, 42, 773-785

Naeimi, V., Bartalis, Z., & Wagner, W. (2008). ASCAT soil moisture: Data quality and consistency with the ERS heritage. Journal Of Hydrometeorology, submitted

Njoku, E. (1996). Nimbus-7 SMMR Pathfinder brightness temperatures. Boulder, CO: National Snow and Ice Data Center

Njoku, E. G., B. Rague, And K. Flemming (1995): Nimbus-7 Scanning Multichannel Microwave Radiometer (SMMR): Brightness Temperature Data (SMMR Level 1B Pathfinder). JPL Publication, Jet Propulsion Laboratory, Pasadena, CA. 'User's Guide for the Scanning Multichannel Microwave Radiometer Instrument First Year Antenna Temperature Data Set', Systems & Applied Sciences Corporation, August 1982, NASA Contract NAS5-27393. 'Nimbus-7 SMMR Pathfinder Brightness Temperatures', NSIDC Dataset Guide Documentation.

Owe M., R. de Jeu and J. Walker, "A methodology for surface soil moisture and vegetation optical depth retrieval using the microwave polarization difference index," in IEEE Transactions on Geoscience and Remote Sensing, vol. 39, no. 8, pp. 1643-1654, Aug. 2001. doi: 10.1109/36.942542

Owe, M., de jeu, R., & Holmes, T. (2008). Multisensor historical climatology of satellite-derived global land surface moisture. Journal of Geophysical Research-Earth Surface, 113, F01002

Owe, M., & Van de Griend, A.A. (1998). Comparison of soil moisture penetration depths for several bare soils at two microwave frequencies and implications for remote sensing. Water Resour. Res., 34, 2319-2327

Parinussa, R, TRH Holmes and RAM de Jeu (2012), Soil moisture retrievals from the WindSat spaceborne polarimetric microwave radiometer. IEEE Trans. Geosci. Remote Sens., DOI: 10.1109/TGRS.2011.2174643

Parinussa, R.M., Meesters, A., Liu, Y.Y., Dorigo, W., Wagner, W., & de Jeu, R.A.M. (2011). Error Estimates for Near-Real-Time Satellite Soil Moisture as Derived From the Land Parameter Retrieval Model. IEEE Geoscience and Remote Sensing Letters, 8, 779-783

Parinussa, R. M., R. H. Thomas, N. W. Holmes, A. D. Wouter, A. M. Richard, and R. A. de Jeu. (2014). “A Preliminary Study toward Consistent Soil Moisture from AMSR2.” Journal of Hydrometeorology 16 (2): 932–947. doi:10.1175/JHM-D-13-0200.1

Peplinski, N.R., Ulaby, F.T., & Dobson, M.C. (1995). Dielectric properties of soils in the 0.3-1.3 GHz range. IEEE Transactions on Geoscience and Remote Sensing, 33, 803-807

Reichle, R.H., Koster, R.D., Dong, J., & Berg, A.A. (2004). Global Soil Moisture from Satellite Observation, Land Surface Models, and Ground Data: Implications for Data Assimilation. Journal of Hydrometeorology, 5, 430-442

Rodell, M. et al., 2004. The Global Land Data Assimilation System. Bulletin of the American Meteorological Society, 85, pp.381–394.

Rui, H., Beaudoing, H., Loeser, C. and Li, B. (2018). README Document for NASA GLDAS Version 2 Data Products. [online] Hydro1.gesdisc.eosdis.nasa.gov. Available at: https://hydro1.gesdisc.eosdis.nasa.gov/data/GLDAS/README_GLDAS2.pdf. URL accessed 04-10-2018.

Schmugge, T.J., O'Neill, P.E., & Wang, J.R. (1986). Passive microwave soil moisture research. IEEE Transaction on Geoscience and Remote Sensing, GE-24, 12-22

Schneeberger, K., Schwank, M., Stamm, C., de Rosnay, P., Matzler, C., & Fluhler, H. (2004). Topsoil structure influencing soil water retrieval by microwave radiometry. Vadose Zone Journal, 3, 1169-1179

Seto, S.; Takahashi, N.; Iguchi, T. Rain/No-Rain Classification Methods for Microwave Radiometer Observations over Land Using Statistical Information for Brightness Temperatures under No-Rain Conditions. J. Appl. Meteorol. 2005, 44, 1243–1259.

TU Wien (2013). ERS AMI-WS (ESCAT) Surface Soil Moisture Product generated from E1/2-SZ-WNF/UWI-00 dataset. Department of Geodesy and Geoinformation, TU Wien, 2013.

Ulaby, F.T., Moore, B., & Fung, A.K. (1982). Microwave Remote Sensing - Active and Passive, Vol. II: Radar Remote Sensing and Surface Scattering and Emission Theory. Norwood: Artech

Uppala, S.M. et al., 2005. The ERA-40 re-analysis. Quarterly Journal of the Royal Meteorological Society, 131(612), pp.2961–3012.

Van der Schalie, R., De Jeu, R.A.M., Kerr, Y.H., Wigneron, J.-P., Rodriguez-Fernandez, N.J., Al-Yaari, A., Parinussa, R.M., Mecklenburg, S., and Drusch, M. (2017), “The merging of radiative transfer based surface soil moisture data from SMOS and AMSR-E”, Remote Sensing of Environment, vol. 189, doi: http://dx.doi.org/10.1016/j.rse.2016.11.026.

Van der Schalie, R., Kerr, Y.H., Wigneron, J.P., Rodriguez-Fernandez, N.J., Al-Yaari, and De Jeu, R.A.M. (2015), “Global SMOS Soil Moisture Retrievals from The Land Parameter Retrieval Model”, Int. J. Appl. Earth Observ. Geoinf., doi: http://dx.doi.org/10.1016/j.jag.2015.08.005.

Van der Schalie, R., Parinussa, R.M., De Jeu, R.A.M., Kerr, Y.H., Wigneron, J.P.,Rodriguez-Fernandez, N.J., Al-Yaari, A., Mecklenburg, S., and Drusch, M. (2018), “The Effect of Three Different Data Fusion Approaches on the Quality of Soil Moisture Retrievals from Multiple Passive Microwave Sensors”, Remote Sensing, vol. 10 (1), no. 107, doi: https://doi.org/10.3390/rs10010107.

Wagner, W. (1998). Soil Moisture Retrieval from ERS Scatterometer Data. In, Institute for Photogrammetry and Remote Sensing. Vienna: Technical University of Vienna

Wagner, W., Lemoine, G., Borgeaud, M., & Rott, H. (1999a). A Study of Vegetation Cover Effects on ERS Scatterometer Data. IEEE Transactions on Geoscience and Remote Sensing, 37,938-948

Wagner, W., Lemoine, G., & Rott, H. (1999b). A Method for Estimating Soil Moisture from ERS Scatterometer and Soil Data. Remote Sensing of Environment, 70, 191-207

Wagner, W., Noll, J., Borgeaud, M., & Rott, H. (1999c). Monitoring soil Moisture over the Canadian Prairies with the ERS Scatterometer. IEEE Trans. Geosci. Rem. Sens., 37, 206-216

Wagner, W., & Scipal, K. (2000). Large-Scale Soil Moisture Mapping in western Africa using the ERS Scatterometer. IEEE Trans. Geosci. Rem. Sens., 38, 1777-1782

Wang, J.R., & Choudhury, B.J. (1981). Remote sensing of soil moisture content over bare field at 1.4 GHz frequency. Journal of Geophysical Research-Oceans and Atmospheres, 86, 5277-5282

Wang, J.R., & Schmugge, T.J. (1980). An empirical model for the complex dielectric permittivity of soils as a function of water content. IEEE Transactions on Geoscience and Remote Sensing,18, 288-295

Wentz, F. J. (1991), User's Manual: SSM/I Antenna Temperature Tapes (Revision 1), report number 120191, Remote Sensing Systems, Santa Rosa, CA, 73 pp. 

Wentz, F. J. (1993), User's Manual: SSM/I Antenna Temperature Tapes (Revision 2), report number 120193, Remote Sensing Systems, Santa Rosa, CA, 36 pp. 

Wentz, F. J., L. Ricciardulli, K. A. Hilburn, and C. A. Mears (2007), How much more rain will global warming bring?, Science, 317, 233-235.

WindSat Data Products Users’ Manual, Sensor & Environmental Data Records, Version 3.0, January 2006, D- 29825, JPL.

Related articles

• Page:
  SM 2020: Target Requirements and Gap Analysis Document (TRGAD)
• Page:
  SM v202012: Algorithm Theoretical Basis Document (ATBD)
• Page:
  SM v201912: Product Quality Assessment Report (PQAR)
• Page:
  SM v 201912: Algorithm Theoretical Basis Document (ATBD)
• Page:
  SM v201812: Algorithm Theoretical Basis Document (ATBD)