...
Depending on the data record producer, different product requirements may be applied and they are used to evaluate validation results. An often-used way to handle this is to define several levels of requirements where each level is linked to specific needs or priorities. A three-level approach like the following is rather common:
Requirement | Description |
|---|---|
Threshold requirement: | A product should at least fulfill this level to be considered |
Target requirement: | This is the main quality goal for a product. It should reach this level based on the current knowledge on what is reasonable to achieve. |
Optimal requirement: | This is a level where a product is considered to perform much better than expected given the current knowledge. |
Satellite product levels
Satellite-based products are often described as belonging to the following condensed description of processing levels, each one with different complexity and information content:
Level | Description |
|---|---|
Level-0: | Raw data coming directly from satellite sensors, often described as sensor counts. |
Level-1: | Data being enhanced with information on calibration and geolocation. Level-1a: Data with attached calibration and geolocation information Level-1b: Data with applied calibration and attached geolocation information Level-1c: Data with applied calibration and additional layers of geolocation, satellite viewing and solar angle information |
Level-2: | Derived geophysical variables at the same resolution and location as L1 source data. An often-used Level-2 variety is the following: Level-2b: Globally resampled images, two per day per satellite, describing both ascending (passing equator from south) and descending (passing equator from north) nodes. Resampling is based on the principle that the value for the pixel with the lowest satellite zenith angle is chosen in case two or several swaths are overlapping. |
Level-3: | Gridded data with results accumulated over time (e.g., monthly means). |
A more comprehensive definition of all processing levels is given here:
...
Table 2‑1: Target requirements for the GRM specific humidity product.Anchor table2_1 table2_1
Variable | KPI: accuracy (Bias) | KPI: decadal stability |
Spec. Hum. | 3 % | - |
The consistency between the ICDR version of the Water Vapour GRM specific humidity product and the corresponding TCDR product is checked by a test designed to detect certain type of differences between the ICDR and the TCDR [D3]. The relative differences between the monthly mean observed data and a reference data set are computed on a global latitude-height grid, for both the ICDR and the TCDR. These relative differences are globally averaged (properly area weighted) and vertically averaged (in 0-4 km, 4-8 km, and 8-12 km layers). For each vertical layer, we find the 2.5% and 97.5% percentiles of the TCDR differences. These percentiles are used in a binomial test to check whether the corresponding ICDR differences are consistent with the TCDR differences. Table 2-2 shows the actual values used for the limit percentiles in the binomial test.
...
Table 2‑2 KPIs for the ICDR of the GRM specific humidity productAnchor table2_2 table2_2
Variable | KPI: lower percentile (2.5 %) | KPI: higher percentile (97.5 %) |
Spec. Hum. | 0-4 km: -2.00 % 4-8 km: -1.04 % 8-12 km: -0.66 % | 0-4 km: -0.93 % 4-8 km: +0.82 % 8-12 km: +2.13 % |
The ICDR used ERA-Interim as a reference data set initially but from August 2019 and onward ERA5 is the reference.
...
Table 2‑3: Comparison of the GRM TCDR target requirements for specific humidity with the GCOS target requirement.Anchor table2_3 table2_3
Requirement | GCOS (Target) | GRM TCDR |
Mean error | < 5% | <3% |
2.1.3 Data format and content issues
...
Table 2‑4:Achieved validation results) for the Water Vapour TCWV WV_cci/CM SAF (COMBI) TCDR v1.0. Results derived from comparisons with AIRS data are chosen as the target accuracies and are marked in bold text.Anchor table2_4 table2_4
Variable | Reference dataset | KPI: accuracy (Bias) | KPI: decadal stability |
TCWV | AIRS ERA5 C3S GOME Evl | 2.5±0.6 % 0.5±0.5 % 0.3±0.4 % 3.1±0.9 % | 0.5±0.4 % 0.7±0.2 % - - |
2.3.1 Discussion of requirements with respect to GCOS and other requirements
...
Table 2‑5: Global comparison of the Water Vapour TCWV WV_cci/CM SAF (COMBI) TCDR v1.0 validation results (where AIRS results, chosen as the target requirement, are marked in bold) with the GCOS target requirement. Marked in green is where the mean error and stability values fulfills the GCOS requirements.Anchor table2_5 table2_5
Requirement | GCOS (Target) | Reference dataset | TCWV WV_cci/CM SAF (COMBI) TCDR |
Spatial resolution | 25 km | - | 5 km |
Temporal resolution | 4h | - | Daily |
Accuracy (mean error) | < 2 % | AIRS | 2.5±0.6 % |
ERA5 | 0.5±0.5 % | ||
C3S | 0.3±0.4 % | ||
GOME Evl | 3.1±0.9 % | ||
Stability | 0.3 % | AIRS | 0.5±0.4 % |
ERA5 | 0.7±0.2 % |
It is noted that the quality over inland water bodies, coastal areas and sea-ice is lower. Depending on the user application, it might be prudent to filter the data accordingly. It was also observed that the transition between MODIS and OLCI based TCWV over land between March and April 2016 is associated with a break point when compared to AIRS and ERA5. Thus, the OLCI period from April 2016 onwards should be excluded from climate change analysis. The NIR based TCWV data over land exhibits a high stability when OLCI data is removed and only clear-sky data is considered. Over ocean a small break point was observed when compared to the merged microwave data record from REMSS. However, the stability is still better than the target product requirement, though not significantly [D2].
...