Contributors: Wouter Dorigo (WD, TU Wien), Tracy Scanlon (TS, TU Wien), Philip Buttinger (PB, TU Wien), Adam Pasik (AP, TU Wien), Wolfgang Preimesberger (WP, TU Wien), Christoph Paulik (CPa, VANDERSAT), Richard Kidd (RK, EODC), Charis Chatzikyriakou (CC, EODC), Tim Ng (TN, EODC)
Issued by: EODC/Richard Kidd
Date: 21/04/2021
Ref: C3S_312b_Lot4.D3.SM.5-v3.0_202101_Product_User_Guide_Specification_i1.0
Official reference number service contract: 2018/C3S_312b_Lot4_EODC/SC2
History of modifications
List of datasets covered by this document
Related documents
Acronyms
Scope of the document
This Product User Guide and Specification PUGS relates to the C3S 312b Lot4 Soil Moisture Climate Data Record (CDR) v202012 and Intermediate Climate Data Record (ICDR) products. It describes the Climate Data Records (CDRs) in a manner that is understood by the product user with focus on the:
- Geophysical data product content
- Known limitations of the product
- Practical Usage Considerations
- Product grid and geographic projection
- Ancillary data used
- Structure and format of the product
- Data file variables and attributes
Executive summary
The CDR and the ICDR are soil moisture Climate Data Records (CDRs) based on the ESA CCI Soil Moisture data product version 4. The C3S Lot4 Soil Moisture products are available at the ECMWF C3S CDS. Both the CDR and the ICDR comprise three data products: The ACTIVE and the PASSIVE products are created by fusing scatterometer and radiometer soil moisture data, respectively; the COMBINED product is a blended product based on the former two products.
All products provide datasets featuring Daily, Dekadal (10-day) mean, and Monthly mean as NetCDF4 images at global scale. The data sets span a time period from November 1978 onwards. While the update policy of CDR is subject to certain criteria, the ICDR represents a consistent extension of the CDR. The generation of the ICDR uses the same algorithms and parameters, which are used to create the CDR. The incremental update of the ICDR takes place every 10 days. The CDR has an annual update cycle and either undergoes an evolution update in response to new merging algorithms, parameters, or new input data sets, or a maintenance update in response to processor maintenance.
The theoretical and algorithmic base of the CDRs is described in [D1], and the products and the applied algorithms are extensively discussed in Dorigo et al. (2017), and in Gruber et al. (2017).
1. Climate Data Records: CDR and ICDR
The ECMWF C3S 312b Lot4 Soil Moisture provides two types of data record: the CDR, and the ICDR. The CDR and ICDR consist of three surface soil moisture data sets: ACTIVE, PASSIVE AND COMBINED.
The ACTIVE and the PASSIVE product are created by using scatterometer, and radiometer soil moisture products, respectively; the COMBINED product is a blended product based on the former two data sets. For each data set the Daily, the Dekadal (10-days) mean, and the Monthly mean are available as NetCDF-4 classic format [D5] and comprise global merged surface soil moisture images at a 0.25 degree spatial resolution.
The theoretical and algorithmic basis of the product are described in [D1]. The Signal to Noise Ratio (SNR) merging algorithm is described in Gruber et al. (2018), and Gruber et al. (2017). An overview of all known errors of the soil moisture datasets is provided in [D2] and in Dorigo et al. (2017). Since this suite of products provided by this C3S service are based upon the scientific products developed in ESA’s Climate Change Initiative Soil Moisture ECV project further background and reference documentation can be found on the CCI Soil Moisture project web site (https://climate.esa.int/en/projects/soil-moisture/).
1.1. CDR
A detailed description of the algorithm for the product generation is provided in [D1]. The underlying algorithm is based on that used in the generation of the ESA CCI SM v05.2 product. In addition, detailed provenance traceability information can be found in the metadata of the product (Section 3.3).
A new version of the CDR will be produced in the following cases:
- Merging algorithm updates
- Processing parameter updates
- Addition of new sensors using an existing algorithm
- Change in input products that make a reprocessing necessary
- Detected product errors requiring processor maintenance or upgrade.
1.2. ICDR
The Intermediate Climate Data Record is a consistent extension of the CDR. The ICDR products are generated every dekad (approx. 10 days) and extend the CDR of the same version as the ICDR. The same algorithm and software processor are used for generating the ICDR products, also input parameter are reused. New near real time observation data from the ASCAT-A/B and AMSR2 and SMOS sensors (Table 23) are processed to extend the ICDR products.
1.3. Product description
Both, the CDR and the ICDR comprise the ACTIVE, PASSIVE, and the COMBINED surface soil moisture data products. For each of these data sets, the Daily, the Dekadal mean, and the Monthly mean are available as global images stored in NetCDF4-classic files following the CF1.8 convention [D5]. The Dekadal files feature a 10-day mean of a month, starting from the 1st to the 10th, from 11th to the 20th, and from 21st to the last day of a month. While the Monthly mean represents the soil moisture mean of each month, the Daily files are created directly through the merging of microwave soil moisture data from multiple satellite instruments (see Table 1). The Dekadal and Monthly means are calculated from these Daily files.
The soil moisture attributes of the Daily files are: day/night flag, satellite orbit mode, instrument operating frequency, sensor, original observation timestamp, soil moisture observation status flag, and soil moisture uncertainty. For the Dekadal and the Monthly mean the frequency band, the used sensor, and the number of observations are attributed to the soil moisture entity (see NetCDF data file variables and attributes).
The detailed the specifications including the geophysical parameters used in the CDR products are described in section 3.
Table 1: CDR / ICDR products and data sets: The mean data sets are calculated from the Daily files, which represent the daily observation derived by merging soil moisture data from multiple microwave sensors.
CDR / ICDR Products | Data sets: Daily / Dekadal mean / Monthly mean | ||||||||
ACTIVE |
| ||||||||
PASSIVE |
| ||||||||
COMBINED |
|
1.3.1. ACTIVE Product
The ACTIVE product is the output of merging scatterometer-based soil moisture data, which are derived from AMI-WS and ASCAT (Metop-A and Metop-B). Please refer to Table 23 for detailed information of the active microwave instruments. The ACTIVE CDR product spans the time period from 1991-08-05 to 2020-12-31, and the ACTIVE ICDR product is available from 2021-01-01 onwards. Table 2 shows the used sensors in the corresponding periods.
Table 2: SNR blending period for the ACTIVE CDR and ICDR products
Sensors | Time Period | CDR |
AMI-WS | 1991-08-05 to 2006-12-31 | CDR |
ASCAT-A | 2007-01-01 to 2015-06-20 | CDR |
ASCAT-A & ASCAT-B | 2015-07-21 to 2018-06-30 | CDR |
ASCAT-A & ASCAT-B | 2018-07-01 onwards | ICDR |
1.3.2. PASSIVE Product
The PASSIVE product merges data from SMMR, SSM/I, TMI, AMSR-E, WindSat, AMSR2, SMOS, and SMAP. The PASSIVE CDR includes soil moisture data from 1978-11-01 to 2020-12-31, whereas the PASSIVE ICDR represents its extension from 2021-01-01 onwards. The blending periods and the used sensors are listed in Table 3.
Table 3: SNR blending period for the PASSIVE CDR and ICDR products.
Sensors | Time Period | CDR |
SMMR | 1978-11-01 to 1987-07-08 | CDR |
SSM/I | 1987-07-09 to 1997-12-31 | CDR |
[SSM/I, TMI, SSM/I]* | 1998-01-01 to 2002-06-18 | CDR |
AMSR-E | 2002-06-19 to 2007-09-30 | CDR |
AMSR-E & WindSat | 2007-10-01 to 2010-01-14 | CDR |
AMSR-E & WindSat & SMOS | 2010-01-15 to 2011-10-04 | CDR |
WindSat & SMOS | 2011-10-05 to 2012-06-30 | CDR |
SMOS & AMSR2 | 2012-07-01 to 2015-03-30 | CDR |
SMOS & AMSR2 & SMAP | 2015-03-31 to 2020-12-31 | CDR |
SMOS & AMSR2 & SMAP | 2021-01-01 onwards | ICDR |
*The [SSM/I, TMI, SSM/I] period is latitudinally divided into [90S, 37S] and [90N, 37N] for SSM/I, and the region in between for TMI.
1.3.3. COMBINED Product
The COMBINED product is generated by merging the ACTIVE and the PASSIVE products, therefore the time span for this product ranges from 1978-11-01 to 2020-12-31. Table 4 shows, that the COMBINED CDR spans the time period from 1978-11-01 to 2020-12-31, and the COMBINED ICDR extends the CDR from 2021-01-01 onwards.
Table 4: SNR blending period for the COMBINED CDR and ICDR products.
Sensors (Active / Passive) | Time Period | CDR |
SMMR | 1978-11-01 to 1987-07-08 | CDR |
SSM/I | 1987-07-09 to 1991-08-04 | CDR |
AMI-WS & SSMI | 1991-08-05 to 1997-12-31 | CDR |
AMI-WS & [SSM/I, TMI, SSM/I]* | 1998-01-01 to 2002-06-18 | CDR |
AMI-WS & AMSRE | 2002-06-19 to 2006-12-31 | CDR |
ASCAT-A & AMSRE | 2007-01-01 to 2007-09-30 | CDR |
ASCAT-A & AMSRE & WindSat | 2007-10-01 to 2010-01-14 | CDR |
ASCAT-A & AMSRE & WindSat & SMOS | 2010-01-15 to 2011-10-04 | CDR |
ASCAT-A & WindSat & SMOS | 2011-10-05 to 2012-06-30 | CDR |
ASCAT-A & SMOS & AMSR2 | 2012-07-01 to 2015-03-30 | CDR |
ASCAT-A & SMOS & AMSR2 & SMAP | 2015-03-31 to 2015-07-20 | CDR |
ASCAT-A & ASCAT-B & SMOS & AMSR2 & SMAP | 2015-07-21 to 2020-12-31 | CDR |
ASCAT-A & ASCAT-B & SMOS & AMSR2 & SMAP | 2021-01-01 onwards | ICDR |
*The [SSM/I, TMI, SSM/I] period is latitudinally divided into [90S, 37S] and [90N, 37N] for SSM/I, and the region in between for TMI.
1.4. Product Target requirements
Table 5 assembles the C3S ECV Soil Moisture product target requirements adopted from the GCOS 2011 target requirements and shows to what extent these requirements are currently met by the latest C3S 312b Lot 4 SM products. As one can see, the CDR and ICDR products currently provided by the system are compliant with C3S target requirements and in many cases even go beyond. Further details on product accuracy and stability are provided in PQAD [D3] (methodology to assess) and PQAR [D4] (assessment).
Table 5: Summary of C3S ECV Soil Moisture requirements, the specification of the current C3S 312b lot 4 products, and the target requirements proposed by the consortium, Green shading indicates target requirement is obtained, Yellow shading indicates target requirement is being approached, Red shading indicates that target requirement is not achieved. Items highlighted in bold show where the target requirement has been exceeded
Requirement | C3S and GCOS target requirements | C3S 312b Lot 4 Products |
Product Specification | ||
Parameter of interest | Surface Soil Moisture | Volumetric Surface Soil Moisture |
Unit | Volumetric (m³/m³) | Volumetric (m³/m³ (passive merged product, combined active +passive merged product); (% of saturation (active merged product) |
Product aggregation | L2 single sensor and L3 merged products | Gridded L2 single sensor products (passive microwave products only); L3 merged active, merged passive, and combined active + passive products |
Spatial resolution | 50 km | 25 km |
Record length | >10 years | >40 years (1978/11 - running present) |
Revisit time | Daily | Daily |
Product accuracy | 0.04 m³/m³ | Variable (0.04-0.10 m³/m³), depending on land cover and climate (current assessment for various climates, land covers and texture classes based on in-situ data shows accuracy to be < 0.1 m³/m³) |
Product stability | 0.01 m³/m³/y | 0.01 m³/m³/y (Assessment indicates stability to be < 0.05 m³/m³/y) |
Quality flags | Not specified | Frozen soil, snow coverage, dense vegetation, retrieval failure, sensor used for each observation, overpass mode, overpass time, RFI |
Uncertainty | Daily estimate, per pixel | Daily estimate, per pixel |
Format Specification | ||
Product spatial coverage | Global | Global |
Product update frequency | Monthly to annual | 10-daily ("extension"), and 6 months ("reprocessing") |
Product format | Daily images, Monthly mean images | Daily images, dekadal (10-day) mean, monthly mean images |
Grid definition | 0.25° | 0.25° |
Projection or reference system | Projection: Geographic lat/lon | Projection: Geographic lat/lon |
Data format | NetCDF, GRIB | NetCDF 4 |
Data distribution system | FTP, WMS, WCF, WFS, OpenDAP | FTP/THREDDS |
Metadata standards | CF, obs4mips | NetCDF Climate and Forecast (CF 1.8) Metadata Conventions; ISO 19115, obs4mips (distributed separately through ESGF) |
Quality standards | QA4ECV | EQC to be implemented |
1.5. Data usage information
The known issues and limitations for the passive and active product generation which underlie CDR and ICDR products are provided as brief points in the following section. All issues and limitations are fully addressed in the product ATBD [D1] and references are provided to the specific ATBD section. Following this a summary of practical usage constraints (resulting from direct user feedback over the course of 6 years of ESA’s CCI SM project) are presented.
1.5.1. Known Limitations for Passive product
The known limitations in deriving soil moisture from passive microwave observations are provided in detail in section 3.1.3 of the ATBD [D1]. It should be noted that these issues do not only apply to the current CDR/ICDR data set release but also to soil moisture retrieval from passive microwave observations in general.
1.5.1.1. Vegetation
Vegetation affects the microwave emission, and under a sufficiently dense canopy the emitted soil radiation will become completely masked by the overlaying vegetation.
Please see section 3.1.3.1 of [D1]
1.5.1.2. Frozen surfaces and snow
Under frozen surface conditions the dielectric properties of the water changes dramatically
Please see section 3.1.3.2 of [D1]
1.5.1.3. Water bodies
Water bodies within the satellite footprint can strongly affect the observed brightness temperature due to the high dielectric properties of water.
Please see section 3.1.3.3 of [D1].
1.5.1.4. Rainfall
Rainstorms during the satellite overpass affect the brightness temperature observation
Please see section 3.1.3.4 of [D1]
1.5.1.5. Radio Frequency interference
Natural emission in several low frequency bands are affected by artificial sources, so called Radio Frequency Interference (RFI).
Please see section 3.1.3.5 of [D1]
1.5.1.6. Using night-time observations only
For the current version of the merged passive product only descending overpasses, corresponding to night-time / early morning observations, were considered. This is because near surface land surface temperature gradients are regarded to be reduced at night leading to more robust retrievals (Owe et al., 2008). However, (Brocca et al., 2011) suggest that for specific land cover types day-time observations may provide more robust retrievals than night-time observations, although the exact causes are still unknown. If day-time observations could be introduced to the blended product, this would significantly increase the observation density.
1.5.1.7. Intercalibration of AMSR-E and AMSR2
As AMSR-E has failed to deliver data since 04 October 2011 the continuity of the passive radiometer data is prolonged by using the WindSat and AMSR2 data sets. The Passive product is extended by using WindSat to bridge the time gap between AMSR-E and AMSR2. An intercalibration technique was developed to adjust WindSat soil moisture to AMSR-E and AMSR2 to the adjusted WindSat data (Parinussa et al., 2015). However, the overlapping period used to compute the calibration constants was very short and needs to be updated using a larger time window. Alternatively, JAXA announced they will make an improved intercalibrated AMSR2 product available, which, if outperforming the empirical intercalibration used so far, will be used to generate level 2 AMSR2 LPRM soil moisture estimates. Within the current product, the first 3 years of AMSR2 are scaled to the last 3 years of AMSR-E. In future, additional SM products might be used to bridge the gap between ASMR-E and AMSR2.
1.5.2. Known Limitations for Active product
The known limitations in deriving soil moisture from active microwave observations are provided in detail in section 8.4.2 of the ESA CCI ATBD [D6]. It should be noted that these issues do not only apply to the current CDR/ICDR data set release but also to soil moisture retrieval from active microwave observations in general.
1.5.2.1. Computation of Slope/Curvature Parameters
Please see Wagner 1999
1.5.2.2. Dry and Wet Crossover Angles
Crossover angles may vary across the globe depending upon the evolution of biomass of a specific vegetation type. Please see section 3.2.3.2 of [D1]
1.5.2.3. Backscatter in Arid Regions
In arid regions or more specifically in desert environments it appears that the dry reference shows seasonal variations, which are assumed to reflect vegetation phenology.
Please see section 3.2.3.1 of [D1]
1.5.2.4. Intercalibration of ERS and ASCAT
The generation of the ERS and ASCAT products is still based on their individual time series. The merged ERS + ASCAT could significantly profit from an appropriate Level 1 intercalibration. Besides improving the quality of the individual measurements this would improve the robustness of the calculation of the dry and wet references.
1.5.2.5. Data gaps
Similar as for the passive products, merging ERS and ASCAT into a merged dataset is based on a strict separation in time. Gaps in ASCAT time series can be potentially filled with ERS observations, although the spatial and temporal overlap between both sensors is limited.
1.5.2.6. Positive SM trend in ACTIVE
ASCAT SM shows an assumed unnatural wetting trend, especially in RFI affected regions (densely populated areas) which also affects the ACTIVE product.
1.5.3. Practical Usage Considerations
Some Practical Usage Considerations are provided in the following section. These considerations result from direct user feedback on the use of the ESA CCI SM product during the period 2011 to 2017 and form the core of the ESA CCI SM product FAQ.
1.5.3.1. Climate trends in general and relative dynamics
Before merging the ACTIVE and PASSIVE products into a COMBINED product we first scale both data sets into the dynamic range of the GLDAS-Noah surface soil moisture fields. We perform this processing step to obtain a final product in absolute volumetric units [m3/m3]. Even though the original dynamics of the remote sensing observations are preserved, this step imposes the absolute values and dynamic range (min-max) of the GLDAS-Noah product on the combined product. As a consequence, the COMBINED product cannot be considered an independent dataset representing absolute true soil moisture. Hence, the statistical comparison metrics like root-mean-square-difference and bias based on our combined dataset are scientifically not meaningful. However, the product can be used as a reference for computing correlation statistics or the unbiased root-mean-square-difference.
1.5.3.2. Temporal availability
In the time period 1978 – 1987, the product is only based on the SMMR radiometer. SMMR had a 24 hr on-off cycle to save power, but this was sometimes changed. For example, in 1986 there is a period with daily observations (they switched the 24 hr on-off cycle off). So, the observation density changes over time. In addition, SMMR observes the Earth surface at 12:00 and 24:00 local solar time, which sometimes leads to a shift of one day for the night-time observations.
1.5.3.3. Spatial availability
For areas with dense vegetation (tropical, boreal forests), strong topography (mountains), ice cover (Greenland, Antarctica, Himalayas), a large fractional coverage of water, or extreme desert areas we are not able to make meaningful soil moisture retrievals. Hence, we mask them (see Table 14).
Especially images of the first years from 1978 onwards show clearly these data stripes. This is a typical characteristic in the observation through satellite microwave instruments. Microwave images from the earth's surface are taken while the satellite is orbiting the earth in fixed paths. These paths represent the data stripes on the images. If we move forward in time, the spatial data availability is getting higher and higher, and the data stripes are getting closer and closer. This is due to the fact that not only the number of available input data sources (satellites) is growing, but also the technology of satellites instruments is getting better and better.
- Some image files do not provide any soil moisture data at all. All values are NaN. We call these images "blank" or "empty" days. Because of many reasons, e.g. technical failures, there is no data available for that day. Especially the SMMR and the AMI-WS (ERS1/2) instruments are known for their data outages causing these blank days. Other instruments also have short time periods with no data availability. In most cases these empty periods are replaced or filled with data from the remaining microwave sensor(s). So blank days are most likely experienced on days where only one sensor is used as input source, which then fails to deliver data for that time.
- When the soil is frozen or covered with snow, we are not able to make a meaningful soil moisture retrieval. Such observations are masked and indicated with flag number 1 in the NetCDF file.
- Based on the sensitivity to vegetation density, we decided for each pixel whether to use either the scatterometer or the radiometer retrievals, or to use a weighted average of the available observations from different sensors. This merging scheme may lead to data gaps in the following situations:
- No observation is available (sensors fail). This is for example the case between 2001 and 2006 in Western Europe, parts of Siberia, parts of North and South America, due to failure of the onboard storage capacity of ERS-2.
- Changes in observation wavelength (frequency) may lead to increased sensitivity to vegetation. Hence, larger areas need to be masked. This is for example visible for the period after 1987 where based on the SSM/I Ku-band observations, the extent of masked areas increases with respect to the preceding SMMR period (C-Band).
1.5.3.4. Data inconsistencies
For AMI-WS and ASCAT soil moisture values may show jumps where ascending and descending swaths overlap with each other, e.g. in the higher northern latitudes. This is a natural phenomenon related to the differences in overpass time (up to 24h). Potentially different soil moisture values may result from precipitation or evaporation taking place between the two observation time steps. We therefore recommend using the original observation time (t0) and not the nominal overpass time if you want to make a direct comparison e.g. with in-situ observations.
1.5.3.5. Data characteristics
The sensors used for each period are best described by the graphic in Figure 1.
Figure 1: Spatial-temporal coverage of input products used to construct the CDR/ICDR (a) ACTIVE, (b) PASSIVE, (c) COMBINED. Blue colours indicate active, red colours passive microwave sensors. The periods of unique sensor combinations are referred to as 'blending period'. Modified from Dorigo et al. (2017).
1.5.3.6. Data usage in models
In theory, the COMBINED product combines the best of the active and passive products, so we consider it as most suitable for model verification.
Only for the mountain ranges in southern Turkey the merged dataset is known to be inferior to the PASSIVE product, see also: Szczypta et al. (2014).
1.5.3.7. Converting volumetric soil moisture in soil wetness content
The equation to convert volumetric soil moisture (SMvol in m3m-3) into degree of saturation (SMsat in %) is the following:
\[ SM (\%) = SM\_vol (m^3m^{-3}) / porosity\_vol (m^3m^{-3}) \] \[ SM_{sat}=\frac{SM_{vol}}{\phi_{vol}} \]Where ϕvol is the soil porosity which can be obtained from soil porosity maps.
1.6. Product Change Log
Table 6 provides an overview of the differences between different versions of the product up-to, and including, the current version.
Table 6: Changes in the product between versions.
Version | Product Changes |
v202012 | The PASSIVE and COMBINED product now include SM data from SMAP brightness temperature measurements derived through the LPRM v6 retrieval model. Intercalibration of AMSRE and AMSR2 observations, that lead to a negative break in PASSIVE SM in the past has been corrected. The CDF matching method has been updated. LPRM v6 is now used to derive SM for all decommissioned passive sensor products. |
v201912 | Product algorithm same as v201812. Product extended to 2019-12-31. |
v201812 | Product algorithm updated and now based on ESA CCI SM v4.4 rather than v4.3 applied to version v201806. Product extended to 2018-12-31. |
v201806 | The combined product is now generated by merging all active and passive L2 products directly, rather than merging the generated active and passive products. Spatial gaps in TC-based SNR estimates now filled using a polynomial SNR_VOD regression. sm_uncertainties now available globally for all sensors except SMMR. A p-value based mask is used to exclude unreliable input data sets in the combined product has been modified and is also applied to the passive product. Masking of unreliable retrievals is undertaken prior to merging. |
v201801 | Updated CDR includes SMOS data from the end of 2016 onwards. SMOS is also included in the ICDRs produced from January 2018 onwards. CDR produced until 2017-12-31; ICDR produced from 2018-01-01. |
v201706 | First release of the dataset. CDR produced until 2017-06-30; ICDR produced from 2017-07-01. |
2. Data access information
2.1. Climate Data Store
The Copernicus Climate Change Service provides data storage infrastructure and make ECV data products available through the CDS. The store provides not only consistent estimates of ECVs, but also climate indicators, and other relevant information about the past, present, and future evolution of the coupled climate system, on global, continental, and regional scales. It supports users with data dissemination and visualisation tools1.
2.2. C3S Soil Moisture data
C3S soil moisture CDRs and ICDRs are available via the CDS.
2.3. User Support
A dedicated service desk has been set up, the Copernicus User Support (CUS) team, which provides support to users of the CAMS and C3S services at ECMWF. All enquiries about the soil moisture dataset must be submitted through the service desk where appropriate agents will deal with it.
There is a forum (https://confluence.ecmwf.int/display/CUSF/forum) where users can browse issues or a knowledge base (https://confluence.ecmwf.int//display/CKB) where customers can also submit direct enquiries. Once submitted, the user may add comments or further information to the issue, including responding to questions / requests for additional information from the support team.
The C3S 312b Lot4 service provides dedicated level 2 user support to the CUS Jira Ticketing Service
3. Specifications for CDR and ICDR
3.1. Geophysical parameters
The ACTIVE product is the output of merging scatterometer-based soil moisture data, which were derived from AMI-WS and ASCAT (Metop-A and Metop-B). The PASSIVE product merges data from SMMR, SSM/I, TMI, AMSR-E, WindSat, AMSR2, SMOS, and SMAP. The COMBINED product merges the ACTIVE and the PASSIVE products. The merging algorithm described here (for ESA CCI SM v05.2) is an evolution of the algorithm described in Dorigo et al. (2017); Liu et al. (2012); Liu et al. (2011); Wagner et al. (2012)), which was used in all previous product versions. The introduced algorithm is described in detail (Gruber et al., 2018). The homogenised and merged products present surface soil moisture with a global coverage and a spatial resolution of 0.25°. The Daily data set has a temporal resolution of 1 day, the Dekadal mean represents a 10-day average of the Daily data, and the Monthly mean performs the averaging of the Daily files for each month. The reference time is set at 0:00 UTC for all products. The soil moisture data for the PASSIVE and the COMBINED product are provided in volumetric units [m3m-3], while the ACTIVE soil moisture data are expressed in percentage of saturation [%].
3.1.1. Product Grid and Projection
The grid is a 0.25° x 0.25° longitude-latitude global array of points, based on the World Geodetic System 1984 (WGS 84) reference system. Its dimension is 1440 x 720, where the first dimension, X (longitude), is incremental from West (-180°) to East (180°), and the second dimension, Y (latitude) is incremental from South (-90°) to North (90°). Grid edges are at multiple of quarter-degree values (e.g. 90.00, 89.75, 89.50, 89.25, …), and the grid centers are exactly between the two grid edges:
First point center = (–89.875°S, –179.875°W) = Grid Point Index = 0
Second point center = (–89.875°S, –179.625°W) = Grid Point Index = 1
…
1441st point center = (–89.625°S, –179.875°W) = Grid Point Index = 1440
…
Last point center = (89.875°N, 179.875°E) = Grid Point Index = 1036799
In total, there are 1440 x 720 = 1036800 grid points, where 244243 points are land points. The land mask has been derived from the Global Self-consistent, Hierarchical, High-resolution Geography Database (GSHHG v2.2.2) (Wessel and Smith, 1996). Lakes and rivers with areas less than 600 km2 were not considered in the calculation of the land points.
Figure 2 shows the land points which are used for each product described in this document.
The Tropical forest mask – derived from the mean AMSR-E VOD (Figure 3) for 2002 to 2011 – has been applied to the soil moisture product images. The soil moisture and the soil moisture uncertainty values are set to NaN in these rainforest regions.
Figure 2: Land mask used for the merged product. The 0.25° grid starts indexing from "lower left" to the "upper right". Note that not all grid points are available for all sensors, e.g. ASCAT retrievals are available between Latitude degrees 80° and –60°.
Figure 3: Tropical forest Mask used applied to the product images. 1 (green) represents rainforest regions.
3.2. Ancillary data
The process of generating the C3S 312b soil moisture products requires the usage of various ancillary data sets. These ancillary datasets are described in the following subsections.
3.2.1. Global Land Data Assimilation System (GLDAS)
The PASSIVE and ACTIVE products represent volumetric soil moisture (m3m-3) and degree of saturation (%), respectively. To combine these data, both products need to be adjusted to a common reference which can be achieved using a reference dataset. The reference dataset requires global coverage with a spatial resolution and temporal interval that are comparable to both of the microwave products (i.e., approximately 25 km resolution and daily interval), a long time record, and reasonable surface soil moisture estimates for all land cover types (i.e., representative soil layer is not deeper than 10 cm).
The GLDAS-Noah v2.1 Land Surface Model L4 3 Hourly 0.25 x 0.25 degree soil moisture model data satisfies these requirements and is employed as the reference dataset. Both (the PASSIVE and ACTIVE) products were rescaled against the GLDAS-Noah data using the CDF matching technique. The methodology behind the use of this data set is provided in [D1].
3.2.2. ASCAT Advisory Flag
The following two ASCAT advisory flags (Scipal, 2005) are used to mask out regions of frozen soils, or snow covered soils:
- Probability of snow covered land
Derived from historic analysis of SSM/I (Special Sensor Microwave/Imager) snow cover data (averaged over the 9 years 1996-2004) and gives the probability for the occurrence of snow for any day of the year.
- Probability of frozen land
Derived from historic analysis of modelled climate data (7 years 1995-2001 of ECMWF ERA-40 soil temperature) and gives the probability for the frozen soil/canopy conditions for each day of the year.
3.2.3. Average Vegetation Optical Depth from AMSR-E
Vegetation optical depth (VOD) estimated from AMSR-E with the VUA-NASA LPRM method are provided to give an indication of vegetation density (Figure 4). The provided global values represent the averaged VOD from 2002 to 2011.
Figure 4: AMSR-E (from LPRM) average vegetation optical depth derived for the period 2002-2011 in the 6.9 GHz band.
3.2.4. Topographic Complexity
The topographic complexity (Normalized standard deviation of topography) is derived from the USGS 30-second Global Elevation Data (GTOPO30) (USGS, 1996). This can be used to help understand the potential distortion of backscatter in mountainous regions (i.e. calibration errors due to the deviation of the surface from the assumed ellipsoid and the rough terrain, the influence of permanent snow and ice cover, a reduced sensitivity due to forest and rock cover and highly variable surface conditions). The topographic complexity flag is derived from GTOPO30 data. For each cell of the DGG, the standard deviation of elevation is calculated, and the result is normalised to values between 0 and 100 % (Figure 5).
Figure 5: Topographic complexity from the USGS 30-second Global Elevation Data (GTOPO30).
3.2.5. Wetland fraction
The open water fraction is defined as fraction coverage of areas with inundation potential. The inundation potential has been derived from the Global Lakes and Wetlands Database (GLWD) level 3 product, which includes several wetland and inundation types. The wetland fraction is calculated for the Discrete Global Grid (DGG)and the conversion from DGG to the 0.25 degree grid is based on the nearest-neighbour search algorithm (Figure 6).
Figure 6: Wetland fraction derived from the Global Lakes and Wetlands Database (GLWD).
3.3. Structure and file format
3.3.1. Data file format and file naming
The file format used for storing the data is NetCDF-4 classic. All (NetCDF) files follow the NetCDF Climate and Forecast (CF) Metadata Conventions version 1.8. The NetCDF soil moisture data files are stored in folders for each year with one file per day. The following file naming convention is applied:
C3S-SOILMOISTURE-L3S-<Variable>-<Dataset>-<Interval>-<Reference_date>-<CDR>-v<Version>.nc
<Variable>
Active product: SSMS (surface soil moisture degree of saturation absolute); Passive and Combined product: SSMV (surface soil moisture volumetric absolute).
<Dataset>
ACTIVE; PASSIVE; COMBINED
<Interval>
DAILY; DEKADAL; MONTHLY
<Reference_date>
YYYYMMDDhhmmss – Reference date and time of the file in UTC. Each daily file contains data from this reference time +- 12 hours. For monthly and dekadal files this reference time is the start of the period. E.g. for the dekadal data the dates can only be YYYYMM01000000, YYYYMM11000000, or YYYYMM21000000. The reference date for the monthly data is always YYYYMM01000000.
<CDR>
Type of Climate Data Record: TCDR; ICDR
v<Version>
Major.Minor.Run e.g. v202012.0.0
The Major number usually represents the year (YYYY) and month (MM) of date. The initial value for Minor is zero and will increment when updating the file. If there is a need – e.g. because of technical issues – to replace a file which already has been made public, the Run number of the replacement file shifts to the next increment. The initial Run number is zero.
3.4. NetCDF global attributes for the ACTIVE products
Table 7: Global NetCDF Attributes for the ACTIVE Daily product
Global Attribute Name | Content |
---|---|
title | C3S Surface Soil Moisture merged ACTIVE Product |
institution | EODC (AUT); TU Wien (AUT); VanderSat B.V. Noordwijk (NL) |
contact | C3S_SM_Science@eodc.eu |
source | WARP 5.5R1.1/AMI-WS/ERS12 Level 2 Soil Moisture; WARP 5.4R1.0/AMI-WS/ERS2 Level 2 Soil Moisture; ASCSMR02/ASCAT/MetOp-A SSM Swath Grid 12.5 km sampling; ASCSMR02/ASCAT/MetOp-B SSM Swath Grid 12.5 km sampling |
history | <date and time auditing trail of modifications to the original data> - file produced |
references | https://climate.copernicus.eu; Liu, Y.Y., Dorigo, W.A., Parinussa, R.M., de Jeu, R.A.M., Wagner, W., McCabe, M.F., Evans, J.P., van Dijk, A.I.J.M. (2012). Trend-preserving blending of passive and active microwave soil moisture retrievals, Remote Sensing of Environment, 123, 280-297, doi: 10.1016/j.rse.2012.03.014; Liu, Y.Y., Parinussa, R.M., Dorigo, W.A., De Jeu, R.A.M., Wagner, W., van Dijk, A.I.J.M., McCabe, M.F., & Evans, J.P. (2011): Developing an improved soil moisture dataset by blending passive and active microwave satellite based retrievals. Hydrology and Earth System Sciences, 15, 425-436; Wagner, W., W. Dorigo, R. de Jeu, D. Fernandez, J. Benveniste, E. Haas, M. Ertl (2012): Fusion of active and passive microwave observations to create an Essential Climate Variable data record on soil moisture. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume I-7, 2012. XXII ISPRS Congress, 25 August - 01 September 2012, Melbourne, Australia; Gruber, A., Dorigo, W. Crow, W., & Wagner, W. (2017). Triple collocation-based merging of satellite soil moisture retrievals. IEEE Transactions on Geoscience and Remote Sensing, 1-13. Dorigo, W.A., Wagner, W., Albergel, C., Albrecht, F., Balsamo, G., Brocca, L., Chung, D., Ertl, M., Forkel, M., Gruber, A., Haas, E., Hamer, P. D., Hirschi, M., Ikonen, J., de Jeu, R., Kidd, R., Lahoz, W., Liu, Y. Y.,Miralles, D., Mistelbauer, T., Nicolai-Shaw, N., Parinussa, R., Pratola, C., Reimer, C., van der Schalie, R., Seneviratne, S. I. Smolander, T., Lecomte, P. (2017). ESA CCI Soil Moisture for improved Earth system understanding: State-of-the art and future directions, Remote Sensing of Environment. https://doi.org/10.1016/j.rse.2017.07.001. |
tracking_id | <xxxxxxxx-yyyy-zzzz-nnnn-mmmmmmmmmmmm> a UUID value |
conventions | CF-1.6 |
product_version | v202012 |
summary | The data set was produced with funding from the Copernicus Climate Change Service. |
keywords | Soil Moisture/Water Content |
id | <filename> |
naming_authority | EODC |
keywords_vocabulary | NASA Global Change Master Directory (GCMD) Science Keywords |
cdm_data_type | Grid |
comment | These data were produced as part of the Copernicus Climate Change Service. Service Contract No 2018/C3S_312b_Lot4_EODC/SC2 |
date_created | <file creation date> |
creator_name | Earth Observation Data Center (EODC) |
creator_url | |
creator_email | C3S_SM_Science@eodc.eu |
project | Copernicus Climate Change Service. |
geospatial_lat_min | -90.0 |
geospatial_lat_max | 90.0 |
geospatial_lon_min | -180.0 |
geospatial_lon_max | 180.0 |
geospatial_vertical_min | 0.0 |
geospatial_vertical_max | 0.0 |
time_coverage_start | <date time start> |
time_coverage_end | <date time end> |
time_coverage_duration | <Daily> P1D; <Dekadal mean>: P10D|P8D|P9D|P10D|P11D; <Monthly mean> : P1M |
time_coverage_resolution | P1D |
standard_name_vocabulary | NetCDF Climate and Forecast (CF) Metadata Convention |
license | Copernicus Data License |
platform | ERS-1, ERS-2, Metop-A, Metop-B |
sensor | AMI-WS, ASCAT-A, ASCAT-B |
spatial_resolution | 25km |
geospatial_lat_units | degrees_north |
geospatial_lon_units | degrees_east |
geospatial_lon_resolution | 0.25 degree |
geospatial_lat_resolution | 0.25 degree |
3.5. NetCDF global attributes for the PASSIVE products
Table 8: Global NetCDF Attributes for the PASSIVE Daily product
Global Attribute Name | Content |
---|---|
title | C3S Surface Soil Moisture merged PASSIVE Product |
institution | EODC (AUT); TU Wien (AUT); VanderSat B.V. Noordwijk (NL) |
contact | C3S_SM_Science@eodc.eu |
source | LPRMv06/SMMR/Nimbus 7 L3 Surface Soil Moisture, Ancillary Params, and quality flags; LPRMv06/SSMI/F08, F11, F13 DMSP L3 Surface Soil Moisture, Ancillary Params, and quality flags; LPRMv06/TMI/TRMM L2 Surface Soil Moisture, Ancillary Params, and QC; LPRMv06/AMSR-E/Aqua L2B Surface Soil Moisture, Ancillary Params, and QC; LPRMv06/WINDSAT/CORIOLIS L2 Surface Soil Moisture, Ancillary Params, and QC; LPRMv06/AMSR2/GCOM-W1 L3 Surface Soil Moisture, Ancillary Params; LPRMv06/SMOS/MIRAS L3 Surface Soil Moisture, CATDS Level 3 Brightness Temperatures (L3TB) version 300 RE03 & RE04 |
history | <date and time auditing trail of modifications to the original data> - file produced |
references | https://climate.copernicus.eu; Liu, Y.Y., Dorigo, W.A., Parinussa, R.M., de Jeu, R.A.M., Wagner, W., McCabe, M.F., Evans, J.P., van Dijk, A.I.J.M. (2012). Trend-preserving blending of passive and active microwave soil moisture retrievals, Remote Sensing of Environment, 123, 280-297, doi: 10.1016/j.rse.2012.03.014; Liu, Y.Y., Parinussa, R.M., Dorigo, W.A., De Jeu, R.A.M., Wagner, W., van Dijk, A.I.J.M., McCabe, M.F., & Evans, J.P. (2011): Developing an improved soil moisture dataset by blending passive and active microwave satellite based retrievals. Hydrology and Earth System Sciences, 15, 425-436; Wagner, W., W. Dorigo, R. de Jeu, D. Fernandez, J. Benveniste, E. Haas, M. Ertl (2012): Fusion of active and passive microwave observations to create an Essential Climate Variable data record on soil moisture. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume I-7, 2012. XXII ISPRS Congress, 25 August - 01 September 2012, Melbourne, Australia; Gruber, A., Dorigo, W. Crow, W., & Wagner, W. (2017). Triple collocation-based merging of satellite soil moisture retrievals. IEEE Transactions on Geoscience and Remote Sensing, 1-13. Dorigo, W.A., Wagner, W., Albergel, C., Albrecht, F., Balsamo, G., Brocca, L., Chung, D., Ertl, M., Forkel, M., Gruber, A., Haas, E., Hamer, P. D., Hirschi, M., Ikonen, J., de Jeu, R., Kidd, R., Lahoz, W., Liu, Y. Y.,Miralles, D., Mistelbauer, T., Nicolai-Shaw, N., Parinussa, R., Pratola, C., Reimer, C., van der Schalie, R., Seneviratne, S. I. Smolander, T., Lecomte, P. (2017). ESA CCI Soil Moisture for improved Earth system understanding: State-of-the art and future directions, Remote Sensing of Environment. https://doi.org/10.1016/j.rse.2017.07.001. |
tracking_id | <xxxxxxxx-yyyy-zzzz-nnnn-mmmmmmmmmmmm> a UUID value |
conventions | CF-1.6 |
product_version | v202012 |
summary | The data set was produced with funding from the Copernicus Climate Change Service. |
keywords | Soil Moisture/Water Content |
id | <filename> |
naming_authority | EODC |
keywords_vocabulary | NASA Global Change Master Directory (GCMD) Science Keywords |
cdm_data_type | Grid |
comment | These data were produced as part of the Copernicus Climate Change Service. Service Contract No 2018/C3S_312b_Lot4_EODC/SC2 |
date_created | <file creation date> |
creator_name | Earth Observation Data Center (EODC) |
creator_url | http://eodc.eu |
creator_email | C3S_SM_Science@eodc.eu |
project | Copernicus Climate Change Service. |
geospatial_lat_min | -90.0 |
geospatial_lat_max | 90.0 |
geospatial_lon_min | -180.0 |
geospatial_lon_max | 180.0 |
geospatial_vertical_min | 0.0 |
geospatial_vertical_max | 0.0 |
time_coverage_start | <date time start> |
time_coverage_end | <date time end> |
time_coverage_duration | <Daily> P1D; <Dekadal mean>: P10D|P8D|P9D|P10D|P11D; <Monthly mean> : P1M |
time_coverage_resolution | P1D |
standard_name_vocabulary | NetCDF Climate and Forecast (CF) Metadata Convention |
license | Copernicus Data License |
platform | Nimbus 7, DMSP, TRMM, AQUA, Coriolis, GCOM-W1, MIRAS, SMAP_ |
sensor | SMMR, SSM/I, TMI, AMSR-E, WindSat, AMSR2, SMOS, SMAP_radiometer |
spatial_resolution | 25km |
geospatial_lat_units | degrees_north |
geospatial_lon_units | degrees_east |
geospatial_lon_resolution | 0.25 degree |
geospatial_lat_resolution | 0.25 degree |
3.6. NetCDF global attributes for the COMBINED products
Table 9: Global NetCDF Attributes for the COMBINED Daily product
Global Attribute Name | Content |
---|---|
title | C3S Surface Soil Moisture COMBINED active+passive Product |
institution | EODC (AUT); TU Wien (AUT); VanderSat B.V. Noordwijk (NL) |
contact | C3S_SM_Science@eodc.eu |
source | WARP 5.5R1.1/AMI-WS/ERS12 Level 2 Soil Moisture; WARP 5.4R1.0/AMI-WS/ERS2 Level 2 Soil Moisture; ASCSMR02/ASCAT/MetOp-A SSM Swath Grid 12.5 km sampling; ASCSMR02/ASCAT/MetOp-B SSM Swath Grid 12.5 km sampling, LPRMv05/SMMR/Nimbus 7 L3 Surface Soil Moisture, Ancillary Params, and quality flags; LPRMv06/SSMI/F08, F11, F13 DMSP L3 Surface Soil Moisture, Ancillary Params, and quality flags; LPRMv06/TMI/TRMM L2 Surface Soil Moisture, Ancillary Params, and QC; LPRMv06/AMSR-E/Aqua L2B Surface Soil Moisture, Ancillary Params, and QC; LPRMv06/WINDSAT/CORIOLIS L2 Surface Soil Moisture, Ancillary Params, and QC; LPRMv06/AMSR2/GCOM-W1 L3 Surface Soil Moisture, Ancillary Params; LPRMv06/SMOS/MIRAS L3 Surface Soil Moisture, CATDS Level 3 Brightness Temperatures (L3TB) version 300 RE03 & RE04 |
history | <date and time auditing trail of modifications to the original data> - file produced |
references | https://climate.copernicus.eu; Liu, Y.Y., Dorigo, W.A., Parinussa, R.M., de Jeu, R.A.M., Wagner, W., McCabe, M.F., Evans, J.P., van Dijk, A.I.J.M. (2012). Trend-preserving blending of passive and active microwave soil moisture retrievals, Remote Sensing of Environment, 123, 280-297, doi: 10.1016/j.rse.2012.03.014; Liu, Y.Y., Parinussa, R.M., Dorigo, W.A., De Jeu, R.A.M., Wagner, W., van Dijk, A.I.J.M., McCabe, M.F., & Evans, J.P. (2011): Developing an improved soil moisture dataset by blending passive and active microwave satellite based retrievals. Hydrology and Earth System Sciences, 15, 425-436; Wagner, W., W. Dorigo, R. de Jeu, D. Fernandez, J. Benveniste, E. Haas, M. Ertl (2012): Fusion of active and passive microwave observations to create an Essential Climate Variable data record on soil moisture. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume I-7, 2012. XXII ISPRS Congress, 25 August - 01 September 2012, Melbourne, Australia; Gruber, A., Dorigo, W. Crow, W., & Wagner, W. (2017). Triple collocation-based merging of satellite soil moisture retrievals. IEEE Transactions on Geoscience and Remote Sensing, 1-13. Dorigo, W.A., Wagner, W., Albergel, C., Albrecht, F., Balsamo, G., Brocca, L., Chung, D., Ertl, M., Forkel, M., Gruber, A., Haas, E., Hamer, P. D., Hirschi, M., Ikonen, J., de Jeu, R., Kidd, R., Lahoz, W., Liu, Y. Y.,Miralles, D., Mistelbauer, T., Nicolai-Shaw, N., Parinussa, R., Pratola, C., Reimer, C., van der Schalie, R., Seneviratne, S. I. Smolander, T., Lecomte, P. (2017). ESA CCI Soil Moisture for improved Earth system understanding: State-of-the art and future directions, Remote Sensing of Environment. https://doi.org/10.1016/j.rse.2017.07.001. |
tracking_id | <xxxxxxxx-yyyy-zzzz-nnnn-mmmmmmmmmmmm> a UUID value |
conventions | CF-1.6 |
product_version | v202012 |
summary | The data set was produced with funding from the Copernicus Climate Change Service. |
keywords | Soil Moisture/Water Content |
id | <filename> |
naming_authority | EODC |
keywords_vocabulary | NASA Global Change Master Directory (GCMD) Science Keywords |
cdm_data_type | Grid |
comment | These data were produced as part of the Copernicus Climate Change Service. Service Contract No 2018/C3S_312b_Lot4_EODC/SC2 |
date_created | <file creation date> |
creator_name | Earth Observation Data Center (EODC) |
creator_url | http://eodc.eu |
creator_email | C3S_SM_Science@eodc.eu |
project | Copernicus Climate Change Service. |
geospatial_lat_min | -90.0 |
geospatial_lat_max | 90.0 |
geospatial_lon_min | -180.0 |
geospatial_lon_max | 180.0 |
geospatial_vertical_min | 0.0 |
geospatial_vertical_max | 0.0 |
time_coverage_start | <date time start> |
time_coverage_end | <date time end> |
time_coverage_duration | <Daily> P1D; <Dekadal mean>: P10D|P8D|P9D|P10D|P11D; <Monthly mean> : P1M |
time_coverage_resolution | P1D |
standard_name_vocabulary | NetCDF Climate and Forecast (CF) Metadata Convention |
license | Copernicus Data License |
platform | Nimbus 7, DMSP, TRMM, AQUA, Coriolis, GCOM-W1, MIRAS, SMAP; ERS-1, ERS-2, METOP-A, METOP-B |
sensor | SMMR, SSM/I, TMI, AMSR-E, WindSat, AMSR2, SMOS, SMAP_radiometer; AMI-WS, ASCAT-A, ASCAT-B |
spatial_resolution | 25km |
geospatial_lat_units | degrees_north |
geospatial_lon_units | degrees_east |
geospatial_lon_resolution | 0.25 degree |
geospatial_lat_resolution | 0.25 degree |
4. NetCDF data file variables and attributes
Lon (Daily, Dekadal, Monthly)
Table 10: Attribute Table for Variable lon
NetCDF Attribute | Description |
standard_name | Longitude |
units | degrees_east |
valid_range | [-180.0, 180.0] |
_CoordinateAxisType | Lon |
Lat (Daily, Dekadal, Monthly)
Table 11: Attribute Table for Variable lat
NetCDF Attribute | Description |
standard_name | Latitude |
units | degrees_north |
valid_range | [-90.0, 90.0] |
_CoordinateAxisType | Lat |
Time (Daily, Dekadal, Monthly)
The reference timestamp of the day is saved in the “time” variable. The data values for the reference time are stored as number of “days since 1970-01-01 00:00:00 UTC.”
Table 12: Attribute Table for Variable time (reference time)
NetCDF Attribute | Description |
standard_name | Time |
units | days since 1970-01-01 00:00:00 UTC |
calendar | Standard |
_CoordinateAxisType | Time |
dnflag (Daily)
The Day or Night Flag specifies, whether the observation(s) occurred at local day (1) or night (2) time. A value of 3 indicates that the data is a result of merging satellite microwave data observed during day as well as during night time. In cases where the information cannot be determined the value is set to 0 (zero).
Table 13: Attribute Table for Variable dnflag, only available in the Daily files
NetCDF Attribute | Description |
long_name | Day / Night Flag |
flag_values | [0, 1, 2, 3] |
flag_meanings | 0 = NaN |
_CoordinateAxes | lat lon time |
_FillValue | 0 (NaN); type: signed byte |
flag (Daily)
Flag values are stored as signed bytes, and the default value (NaN) is 127. By reading the flag for the surface soil moisture data, the user gets information for that grid point. A “0” (zero) informs the user that the sm value for that grid point has been checked, but there was no inconsistency found. A flag value of “1” denotes, that the soil for that location is covered with snow or the temperature is below zero;“2” indicates that the observed location is covered by dense vegetation; “4” stands for undefined other cases, e.g. no convergence in the model, thus no valid soil moisture estimates; “8” denotes days that are masked because not all data sets have valid observations and those which do are deemed unreliable when used alone; and “16” denotes locations where all data sets are deemed unreliable. Please see Table 14 for the meaning of all other flag values.
Table 14: Attribute Table for Variable flag, only available in the Daily files
NetCDF Attribute | Description |
long_name | Flag |
flag_values | [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49. 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 127] |
flag_meanings | 0 = no data inconsistency detected 33 = combination of flag values 1 and 32 34 = combination of flag values 2 and 32 35 = combination of flag values 1 and 2 and 32 36 = combination of flag values 4 and 32 37 = combination of flag values 1 and 4 and 32 38 = combination of flag values 2 and 4 and 32 39 = combination of flag values 1 and 2 and 4 and 32 40 = combination of flag values 8 and 32 41 = combination of flag values 1 and 8 and 32 42 = combination of flag values 2 and 8 and 32 43 = combination of flag values 1 and 2 and 8 and 32 44 = combination of flag values 4 and 8 and 32 45 = combination of flag values 1 and 4 and 8 and 32 46 = combination of flag values 2 and 4 and 8 and 32 47 = combination of flag values 1 and 2 and 4 and 8 and 32 48 = combination of flag values 16 and 32 49 = combination of flag values 1 and 16 and 32 50 = combination of flag values 2 and 16 and 32 51 = combination of flag values 1 and 2 and 16 and 32 52 = combination of flag values 4 and 16 and 32 53 = combination of flag values 1 and 4 and 16 and 32 54 = combination of flag values 2 and 4 and 16 and 32 55 = combination of flag values 1 and 2 and 4 and 16 and 32 56 = combination of flag values 8 and 16 and 32 57 = combination of flag values 1 and 8 and 16 and 32 58 = combination of flag values 2 and 8 and 16 and 32 59 = combination of flag values 1 and 2 and 8 and 16 and 32 60 = combination of flag values 4 and 8 and 16 and 32 61 = combination of flag values 1 and 4 and 8 and 16 and 32 62 = combination of flag values 2 and 4 and 8 and 16 and 32 63 = combination of flag values 1 and 2 and 4 and 8 and 16 and 32 |
_CoordinateAxes | lat lon time |
_FillValue | 127 (NaN); type: signed byte |
freqbandID (Daily, Dekadal, Monthly)
The surface soil moisture data has its sources from multiple and different satellite sensors, which operate in various frequencies. The freqbandID values are representing the operating frequencies and comprise the combination of different frequency bands. Table 15 lists these combinations.
Table 15: Attribute Table for Variable freqbandID
NetCDF Attribute | Description | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
long_name | Frequency Band Identification | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
flag_values | [0, 1, 2, 3, 4, 8, 9, 10, 11, 16, 17, 18, 19, 24, 25, 26, 27, 32, 33, 34, 35, 64, 65, 66, 67, 72, 73, 74, 75, 80, 81, 82, 83, 128, 130] | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
flag_meanings | Flag values and their meaning
List of major codes and the corresponding frequency bands
Sensors and their operating frequencies:
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
_CoordinateAxes | lat lon time | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
_FillValue | 0 (NaN); type: signed integer |
mode (Daily)
The NetCDF variable mode stores the information of the sensor’s orbit direction. Ascending direction are denoted as 1, and descending orbit as 2. In cases where the orbit direction cannot be determined, the NaN value 0 (zero) is used. A value of 3 means that the merged data comprises both ascending and descending satellite modes.
Table 16: Attribute Table for Variable mode
NetCDF Attribute | Description |
long_name | Satellite Mode |
flag_values | [0, 1, 2, 3] |
flag_meanings | 0 = NaN |
_CoordinateAxes | lat lon time |
_FillValue | 0 (NaN); type: signed byte |
nobs (Dekadal, Monthly)
The NetCDF variable nobs stores an integer which is the number of valid observations which have been used to compute the dekadal or monthly mean.
Table 17: Attribute Table for Variable nobs
NetCDF Attribute | Description |
long_name | Number of valid observations |
units | N/A |
_CoordinateAxes | lat lon time |
_FillValue | -1 (NaN); type: short integer |
sensor (Daily, Dekadal, Monthly)
The values for sensor are stored as signed integer, with NaN as 0 (zero). These values indicate the satellite sensors which have been used for a specific grid point. Valid values range from 1 to 1888. Table 18 lists all available sensor combinations.
Table 18: Attribute Table for Variable sensor
NetCDF Attribute | Description | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
long_name | Sensor | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
flag_values | [0, 1, 2, 4, 8, 16, 24, 32, 64, 72, 80, 88, 96, 128, 130, 132, 136, 256, 264, 272, 280, 288, 320, 328, 336, 344, 352, 512, 544, 576, 608, 768, 800, 832, 864, 1024, 1056, 1088, 1120, 1280, 1312, 1344, 1376, 1536, 1568, 1600, 1632, 1792, 1824, 1856, 1888] | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
flag_meanings |
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
_CoordinateAxes | lat lon time | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
_FillValue | 0 (NaN); type: signed integer |
sm (Daily, Dekadal, Monthly)
The “sm” parameter holds the surface soil moisture estimates are generated by blending passive and active microwave soil moisture retrievals as a weighted average with the weights being proportional to the signal-to-noise ratio (SNR) of the data sets. SNRs are estimated using triple collocation (TC) analysis (Gruber et al., 2017). The data are provided in percentage of saturation [%] units for the ACTIVE product, and volumetric [m3m-3] units for the PASSIVE and COMBINED products. Figure 7 shows a plotted example of the sm variable.
Table 19: Attribute Table for Variable sm for the PASSIVE and COMBINED products
NetCDF Attribute | Description |
long_name | ACTIVE: Percent of Saturation Soil Moisture |
units | ACTIVE: percent |
_CoordinateAxes | lat lon time |
_FillValue | -9999.0 (NaN); type: float32 (4 bytes) |
Figure 7: Visualisation of the NetCDF data variable “sm” for day 2017-06-21 from the COMBINED CDR product (from v201801).
sm_uncertainty (Daily, Dekadal, Monthly)
The merging of soil moisture data from different sensors requires a harmonization of the data. The data need to be brought into a common climatology by running them through several scaling procedures performing the cumulative distribution function (CDF) matching technique. The provided “sm_uncertainty” parameter represents the error standard deviation of the data sets (in the respective climatology of the dataset), estimated through triple collocation (TC) analysis, which are used to calculate the relative weighting of the data sets. In periods where TC cannot be applied, or in cases where the TC-based error standard deviation estimates do not converge, sm_uncertainty is set to NaN. The unit of sm_uncertainty for the ACTIVE product is percentage of saturation [%]. For the PASSIVE and the COMBINED product the unit is volumetric soil moisture [m3m-3]. On days where only measurements of one single data set are available, sm_uncertainty represents their error standard deviation as obtained from TC analysis. On days where two or more data sets are merged, sm_uncertainties represents the estimated error standard deviation of the merged soil moisture measurements, obtained by propagating the TC-based error standard deviation estimates of the contributing data sets through the merging algorithm using a standard error propagation scheme. sm_uncertainty values exceeding the maximum value of 100 (ACTIVE) or 1 (PASSIVE and COMBINED) are set to the maximum value respectively. Table 21 lists the availability of the soil moisture uncertainty information for each product. Figure 8 plots the uncertainty for day 2017-06-21 of the CDR COMBINED product.
Table 20: Attribute Table for Variable sm_uncertainty
NetCDF Attribute | Description |
long_name | ACTIVE: Percent of Saturation Soil Moisture Uncertainty |
Units | ACTIVE: percent |
_CoordinateAxes | lat lon time |
_FillValue | -9999.0 (NaN); type: float32 (4 bytes) |
Table 21: sm_uncertainty data provided in the Daily data sets
Product | Time Period |
ACTIVE | 1991-08-05 onwards |
PASSIVE | 1987-07-09 onwards |
COMBINED | 1987-07-09 onwards |
Figure 8: Visualisation of the NetCDF data variable "sm_uncertainty" for day 2017-06-21 from the COMBINED CDR (from v201801).
t0 (Daily)
The original observation timestamp is stored within the NetCDF variable t0 (t-zero). Time values coming from two different sensors are averaged. Values of -9999.0 are used as NaN values. t0 data values are stored as number of "days since 1970-01-01 00:00:00 UTC."
Table 22: Attribute Table for Variable t0
NetCDF Attribute | Description |
long_name | Observation Time Stamp |
units | days since 1970-01-01 00:00:00 UTC |
valid_range | <individual decimal numbers depending on observation timestamp> |
_CoordinateAxes | lat lon time |
_FillValue | -9999.0; type: double |
5. Overview of EO and modelled data used to create the C3S products
Table 23: Major characteristics of passive and active microwave instruments and model products
Passive microwave products | Active microwave products | Model product | |||||||||||||
SMMR | SSM/I | TMI | AMSR-E | AMSR2 | WindSat | SMOS | SMAP | AMI-WS | AMS-WS | ASCAT | ASCAT | GLDAS-2-Noah | GLDAS-2-Noah | ||
Platform | Nimbus 7 | DMSP | TRMM | Aqua | GCOM-W1 | Coriolis | SMOS | SMAP | ERS1/2 | ERS2 | MetOp-A | MetOp-B | — | — | |
Product | VUA NASA | VUA NASA | VUA NASA | VanderSat | VanderSat | VUA NASA | VanderSat | VanderSat | SSM Product (TU WIEN 2013) | SSM Product (Crapolicchio et al. 2016) | H 113/114 (H SAF 2018a and 2018b) | H 113/114 (H SAF 2018a and 2018b) | — | — | |
Algorithm Product version | REF[1] | REF[1] | REF[1] | REF[1] | REF[1] | REF[1] | REF[1] | REF[1] | TU WIEN Change Detection (Wagner et al. 1999b) | TU WIEN Change Detection (Wagner et al. 1999b) | TU WIEN Change Detection (H SAF 2018 c) | TU WIEN Change Detection (H SAF 2018 c) | V2.1 | V2.0 | |
Time period used | Jan 1979 –Aug 1987 | Sep 1987 – Dec 2007 | Jan 1998 – Dec 2013 | Jul 2002 – Oct 2011 | May 2012 – present | Oct 2007 –Jul 2012 | Jan 2010 –present | 03/2015-present | Jul 1991 – Dec 2006 | May 1997 – Feb 2007 | Jan 2007 – | Jul 2015 - present | Jan 2000 – | Jan 1948 – | |
Channel used for soil moisture | 6.6 GHz | 19.3 GHz | 10.7 GHz | 6.9/10.7 GHz | 6.925/10.65 GHz | 6.8/10.7 GHz | 1.4 GHz | 1.4 GHz | 5.3 GHz | 5.3 GHz | 5.3 GHz | 5.3 GHz | — | — | |
Original spatial resolution (km2)* | 150×150 | 69 × 43 | 59 × 36 | 76 × 44 | 35 x 62 | 25 x 35 | 40 km | 38x49 | 50 × 50 | 25 x 25 | 25 × 25 | 25 × 25 | 25 × 25 | 25 × 25 | |
Spatial coverage | Global | Global | N40o to S40o | Global | Global | Global | Global | Global | Global | Global | Global | Global | Global | Global | |
Swath width (km) | 780 | 1400 | 780/897 after boost in Aug 2001 | 1445 | 1450 | 1025 | 600 | 1000 | 500 | 500 | 1100 (550×2) | 1100 (550×2) | — | — | |
Equatorial crossing time | DESC: | DESC: | Varies (non polar-orbiting) | DESC: | DESC: | DESC: | ASC: | DESC: | DESC: | DESC: | DESC: | DESC: | — | — | |
Unit | m3m-3 | m3m-3 | m3m-3 | m3m-3 | m3m-3 | m3m-3 | m3m-3 | m3m-3 | Degree of saturation (%) | Degree of saturation (%) | Degree of saturation (%) | Degree of saturation (%) | kg m-2 | kg m-2 |
References
BROCCA, L., HASENAUER, S., LACAVA, T., MELONE, F., MORAMARCO, T., WAGNER, W., DORIGO, W., MATGEN, P., MARTÍNEZ-FERNÁNDEZ, J., LLORENS, P., LATRON, J., MARTIN, C. & BITTELLI, M. 2011. Soil moisture estimation through ASCAT and AMSR-E sensors: An intercomparison and validation study across Europe. Remote Sensing of Environment, submitted.
DORIGO, W., WAGNER, W., ALBERGEL, C., ALBRECHT, F., BALSAMO, G., BROCCA, L., CHUNG, D., ERTL, M., FORKEL, M., GRUBER, A., HAAS, E., HAMER, P. D., HIRSCHI, M., IKONEN, J., DE JEU, R., KIDD, R., LAHOZ, W., LIU, Y. Y., MIRALLES, D., MISTELBAUER, T., NICOLAI-SHAW, N., PARINUSSA, R., PRATOLA, C., REIMER, C., VAN DER SCHALIE, R., SENEVIRATNE, S. I., SMOLANDER, T. & LECOMTE, P. 2017. ESA CCI Soil Moisture for improved Earth system understanding: State-of-the art and future directions. Remote Sensing of Environment.
GRUBER, A., DORIGO, W., CROW, W. & WAGNER, W. 2017. Triple Collocation-Based Merging of Satellite Soil Moisture Retrievals. IEEE Transactions of Geoscience and Remote Sensing.
GRUBER, A., SCANLON, T., VAN DER SCHALIE, R., WAGNER, W. & DORIGO, W. 2018. Improved merging of multi-satellite products into a soil moisture climate data record. In Prep.
H SAF (2018a). ASCAT Surface Soil Moisture CDR2017 time series 12.5 km sampling – Metop (H113), EUMETSAT SAF on Support to Operational Hydrology and Water Management. http://dx.doi.org/10.15770/EUM_SAF_H_0005
H SAF (2018b). ASCAT Surface Soil Moisture CDR2017-EXT time series 12.5 km sampling – Metop (H114), EUMETSAT SAF on Support to Operational Hydrology and Water Management. https://navigator.eumetsat.int/product/EO:EUM:DAT:METOP:H114
H SAF (2018c) Algorithm Theoretical Baseline Document (ATBD), Metop ASCAT Soil Moisture Data Records v0.7 (H113) http://hsaf.meteoam.it/documents/ATDD/ASCAT_SSM_CDR_ATBD_v0.7.pdf
LIU, Y. Y., DORIGO, W. A., PARINUSSA, R. M., DE JEU, R. A. M., WAGNER, W., MCCABE, M. F., EVANS, J. P. & VAN DIJK, A. I. J. M. 2012. Trend-preserving blending of passive and active microwave soil moisture retrievals. Remote Sensing of Environment, 123, 280-297.
LIU, Y. Y., PARINUSSA, R. M., DORIGO, W. A., DE JEU, R. A. M., WAGNER, W., VAN DIJK, A., MCCABE, F. M. & EVANS, J. P. 2011. Developing an improved soil moisture dataset by blending passive and active microwave satellite-based retrievals. Hydrology and Earth System Sciences, 15, 425-436.
OWE, M., DE JEU, R. & HOLMES, T. 2008. Multisensor historical climatology of satellite-derived global land surface moisture. Journal of Geophysical Research-Earth Surface, 113, F01002.
PARINUSSA, R. M., HOLMES, T. R. H., WANDERS, N., DORIGO, W. & DE JEU, R. A. M. 2015. A preliminary study towards consistent soil moisture from AMSR2. Journal of Hydrometeorology, 16, 932–947.
SCIPAL, K., NAEIMI, V., HASENAUER, S. 2005. Definition of Quality Flags. ASCAT Soil Moisture Report Series. Vienna: Vienna University of Technology.
SZCZYPTA, C., CALVET, J.-C., MAIGNAN, F., DORIGO, W., BARET, F. & CIAIS, P. 2014. Suitability of modelled and remotely sensed essential climate variables for monitoring Euro-Mediterranean droughts Geosci. Model Dev., 7.
TU WIEN (2013). ERS AMI-WS (ESCAT) Surface Soil Moisture Product generated from E1/2-SZ-WNF/UWI-00 dataset. Department of Geodesy and Geoinformation, TU Wien, 2013.
USGS 1996. Global 30-Arc-Second Elevation Data Set (GTOPO30). U.S. Geological Survey.
VAN DER SCHALIE, R., KERR, Y.H., WIGNERON, J.P., RODRIGUEZ-FERNANDEZ, N.J., AL-YAARI, and DE JEU, R.A.M. (2015)), "Global SMOS Soil Moisture Retrievals from The the Land Parameter Retrieval Model", Int. J. Appl. Earth Observ. Geoinf., doi: http://dx.doi.org/10.1016/j.jag.2015.08.005.
WAGNER, W., DORIGO, W., DE JEU, R., FERNANDEZ, D., BENVENISTE, J., HAAS, E. & ERTL, M. 2012. Fusion of Active and Passive Microwave Observations To Create an Essential Climate Variable Data Record on Soil Moisture. ISPRS Annals of Photogrammetry, Remote Sensing and Spatial Information Sciences, I-7, 315--321.
WAGNER, W., LEMOINE, G., & ROTT, H. (1999). A method for estimating soil moisture from ERS scatterometer and soil data. Remote Sensing of Environment, 70, 191-207
WESSEL, P. & SMITH, W. H. F. 1996. A global, self‐consistent, hierarchical, high‐resolution shoreline database. J. Geophys. Res., 10, 8741-8743.
This document has been produced in the context of the Copernicus Climate Change Service (C3S).
The activities leading to these results have been contracted by the European Centre for Medium-Range Weather Forecasts, operator of C3S on behalf of the European Union (Delegation agreement signed on 11/11/2014). All information in this document is provided "as is" and no guarantee or warranty is given that the information is fit for any particular purpose.
The users thereof use the information at their sole risk and liability. For the avoidance of all doubt , the European Commission and the European Centre for Medium - Range Weather Forecasts have no liability in respect of this document, which is merely representing the author's view.