...

Acronyms

List of tables

List of figures

General definitions

Scope of the document

...

The validation methods are described in the beginning of Section 4 of the ROM SAF Level 3 Validation Report [D2], and the actual requirements are stated in Section 6 and in Annex A of the same document.

Anchor
validation validation

2. Validation results

The main validation results are given in Section 5.2 (Figure 22a) of the ROM SAF Level 3 Validation Report [D2]. Additional validation results, relevant for the quality assessment are shown in Section 4.2.2 (Figure 8e) and in Section 5.1 (Figure 20e) of the same report [D2].

...

3. Application(s) specific assessments

In addition to the extensive product validation (see chapter 2 for results and chapter 2/3 in [D6] for validation methodology) a second assessment is introduced to evaluate the Interim Climate Data Record (ICDR) against the Thematic Climate Data Record (TCDR) in terms of consistency. Since frequent ICDR deliveries make detailed validation not feasible, a consistency check against the deeply validated TCDR is used as an indication of quality. This is done by a comparison of the following two evaluations:

• TCDR against a stable, long-term and independent reference dataset
• ICDR against the same stable, long-term and independent reference dataset

The evaluation method is generated to detect differences in the ICDR performance in a quantitative, binary way with so called Key Performance Indicators. The general method is outlined in [D5] chapter 3. The same difference between TCDR/ICDR and the reference dataset would lead to the conclusion that TCDR and ICDR have the same quality (key performance is "good"). Variations or trends in the differences (TCDR/ICDR against reference) would require a further investigation to analyze the reasons. The key performance would be marked as "bad". The binary decision whether the key performance is good or bad is made in a statistical way by a hypotheses test (binomial test). Based on the TCDR/reference comparison (global means, monthly or daily means) a range is defined with 95% of the differences are within. This range (2.5 and 97.5 percentile) is used for the ICDR/reference comparison to check whether the values are in or out of the range. The results could be the following:

• All or a sufficient high number of ICDR/reference differences lies within the range defined by the TCDR/reference comparison: Key performance of the ICDR is "good"
• A smaller number of ICDR/reference differences is within the pre-defined range: Key performance of the ICDR is "bad"

3.1 Results

The results of the KPI test are summarized in Table 3-1.

Percentiles were calculated based on the comparison of the TCDR against ERA-Interim as reference dataset for the tropospheric humidity in three different height levels: 0-4 km, 4-8 km and 8-12 km.

Especially the lowest atmospheric layer ICDR months are mostly outside the pre-defined percentiles and result in "bad" KPI tests. This is mainly related to a known bias shift in the reference data (ERA-Interim). There was a change in the ERA-Interim data assimilation in 2017, which was close to the TCDR/ICDR transition. In addition, the reference data was changed in August 2019 from ERA-Interim to ERA5 leading to differences without having a significant impact. The ICDR becomes closer to compliance with a de-trend before the statistical tests, again without beeing on a significant level. The next reprocessing of all available Metop data is planned to be released by ROM SAF in 2024 and expected to solve the inhomogeneities between TCDR and ICDRThis section is not applicable.

4. Compliance with user requirements

...

References are found in Section 1.2 of the ROM SAF Level 3 Validation Report [D2] and in Section 1.2 in the ROM SAF ICDR Validation Report [D3].

Related articles