...
Expand | ||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ||||||||||||||
|
Acronyms
Expand | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
...
Expand | ||
---|---|---|
| ||
Figure 2-1. : CFC, CTP, LWP and IWP from SLSTR (ICDR dataset) for March 2017 Figure 2-2. : CFC, CTP, LWP and IWP from MODIS dataset for March 2017 |
...
Expand | ||
---|---|---|
| ||
Table 2-1: Summary of the TCDR accuracy and stability of the Cloud Properties dataset (together with the GCOS requirements) extracted from table 7-2 in [D1] Table 2-2: Bias of TCDR and ICDR cloud properties estimate in comparison with MODIS Table 3-1: Summary of KPI results with 2.5 and 97.5 percentiles and number of ICDR months within the range |
General definitions
The "CCI product family" Climate Data Record (CDR) consists of two parts. The ATSR2-AATSR Cloud Properties CDR is formed by a TCDR brokered from the ESA Cloud_cci project and an ICDR derived from the SLSTR on board of Sentinel-3. ICDR uses the same processing and infrastructure as the TCDR. Both TCDR and ICDR data have been produced by STFC RAL Space.
...
Table 1: Summary of variables and definitions
Variables | Abbreviation | Definition |
Cloud mask / Cloud fraction | CMA/ | A binary cloud mask per pixel (L2) and from there derived monthly total cloud fractional coverage (L3C) |
Cloud optical thickness | COT | The line integral of the absorption extinction coefficient (at 0.55μm wavelength) along the vertical in cloudy pixels. |
Cloud effective radius | CER | The area-weighted radius of the cloud droplet and crystal particles, respectively. |
Cloud top pressure/ | CTP/ | The air pressure [hPa] /height [m] /temperature [K] of the uppermost cloud layer that could be identified by the retrieval system. |
Cloud liquid water path/ | LWP/ | The vertical integrated liquid/ice water content of existing cloud layers; derived from CER and COT. LWP and IWP together represent the cloud water path (CWP) |
Table 2: Definition of processing levels
Processing level | Definition |
Level-1b | The full-resolution geolocated radiometric measurements (for each view and each channel), rebinned onto a regular spatial grid. |
Level-2 (L2) | Retrieved cloud variables at full input data resolution, thus with the same resolution and location as the sensor measurements (Level-1b). |
Level-3C (L3C) | Cloud properties of Level-2 orbits of one single sensor combined (averaged) on a global spatial grid. Both daily and monthly products provided through C3S are Level-3C. |
Table 3: Definition of various technical terms used in the document
Term | Definition |
---|---|
Brokered product | The C3S Climate Data Store (CDS) provides both datasets produced within the C3S and so-called brokered products. The latter are existing products (data) produced under an independent programme or project which are made available through the CDS. |
TCDR | It is a consistently-processed time series of a geophysical variable of sufficient length and quality. |
ICDR | An Interim Climate Data Record (ICDR) denotes an extension of TCDR, processed with a processing system as consistent as possible to the generation of TCDR. |
CDR | A Climate Data Record (CDR) is defined as a time series of measurements with sufficient length, consistency, and continuity to determine climate variability and change. |
Scope of the document
This document provides a description of the product validation results for the Essential Climate Variable (ECV) Cloud Properties. This CDR comprises inputs from two sources: (i) brokered products from the Cloud Climate Change Initiative (ESA's Cloud_cci), namely those coming from processing of the Advanced Along-Track Scanning Radiometer (A)ATSR) data and (ii) those produced under this contract for the Climate Data Store, specifically those coming from processing of the Sea and Land Surface Temperature Radiometers (SLSTR).
...
The validation methodology is described in section 2.4 of [D1]. In addition, the validation methodology is also described in the corresponding Product Quality Assurance Document (PQAD) [D4]. In summary, the methodology uses the bias between the Cloud_cci product and the reference data to estimate the accuracy of the dataset. The bias corrected root mean squared error (bc-RMSE) is used to express the precision of CDR compared to a reference data record, which is also known as the standard deviation about the mean. Stability is calculated as the variation of the bias over a multi-annual time period.
Anchor | ||||
---|---|---|---|---|
|
2.1 Validation results for the TCDR
The validation results for TCDR products are provided in [D1], section 3 and 4. The evaluation is divided in validation against high quality and satellite-based reference observations Please find information on the sensors in [D1]: CALIOP (Annex A.1), AMSR-E (A.2), DARDAR (A.3) and MODIS-C6.1 (A.10) (CALIOP, AMSR-E and DARDAR) and an intercomparison to well-established, satellite-based cloud datasets of similar kind (e.g. MODIS Collection 6.1). Table 2-1 provides a summary of the resulting TCDR accuracies.
Table 2-1: Summary of the TCDR accuracy and stability of the Cloud Properties dataset (together with the GCOS requirements) extracted from table 7-2 in [D1]. Green shaded cell indicate compliance with the requirements, yellow cells nearly compliance and red cell no compliance. Anchor table2_1 table2_1
Product name | GCOS targets | Cloud CCI dataset | Comments | |
Cloud Fractional Cover (CFC) | Accuracy | 5 % | -5.1 % | Level-2 validation against CALIOP |
Stability (per decade) | 3 % | -0.52 % | Level – 3C (L3C) comparisons to MODIS C6.1 | |
Cloud Top Height (CTH)/ Pressure (CTP) | Accuracy (low/mid/high) | 0.5/0.7/ 1.6 km | 0.12 km (liquid cloud) | Level-2 validation against CALIOP |
Stability (per decade) | 15 hPa | 0.45 hPa | Level-3C (L3C) comparisons to MODIS C6.1 | |
Cloud Optical Thickness (COT) | Accuracy | 10 % | No validation (n/v) | No validation possible due to a lack of reliable reference data. Used to estimate LWP and IWP, that are validated. |
Stability (per decade) | 2 % | -0.03 % (liquid cloud) | L3C comparisons to MODIS C6.1 | |
Liquid Water Path (LWP) | Accuracy | 25 % | -2.4% | Level-2 validation against AMSR-E |
Stability (per decade) | 5 % | -0.06 %1 | L3C comparisons to MODIS C6.1 | |
Ice Water Path (IWP) | Accuracy | 25 % | -39.9 % | Level-2 validation against DARDAR |
Stability (per decade) | 5 % | -0.04 %2 | L3C comparisons to MODIS C6.1 | |
Cloud Effective Radius (CER) | Accuracy | 10 % | n/v | No validation possible due to a lack of reliable reference data. Variable is based on LWP and IWP. |
Stability (per decade) | 1μm | -0.96 μm (liquid) | L3C comparisons to MODIS C6.1 |
1 Value obtained from the difference between CDR LWP trend (0.99g/m2/decade) and MODIS LWP trend (8.06g/m2/decade) divided by mean MODIS C6.1 LWP (123g/m²).
2 Value obtained from the difference between CDR IWP trend (-2.27g/m2/decade) and MODIS IWP trend (7.82g/m2/decade) divided by mean MODIS C6.1 IWP (208g/m²).
2.1.1 Cloud Fractional Cover (CFC)
...
Table 2-2: Bias of TCDR and ICDR cloud properties estimate in comparison with MODIS. (Dataset till June 2022) Anchor table2_2 table2_2
Parameters | TCDR (2003-2011) bias | ICDR (2017-2022) bias | ICDR (2019-2022) bias | ICDR merged bias (SLSTR-A and B) |
CFC | -8.1% | -6% | -6% | -6% |
CTP | -25 hPa | -15 hPa | -13 hPa | -13 hPa |
LWP | -17.3 g/m2 | -9.0 g/m2 | -1.1 g/m2 | 0.67 g/m2 |
IWP | -28.8 g/m2 | -25 g/m2 | -40 g/m2 | -25 g/m2 |
Figure 2-1: CFC, CTP, LWP and IWP from SLSTR (ICDR dataset) for March 2017, (areas that have no data are shown in white). Anchor figure2_1 figure2_1
Figure 2-2. CFC, CTP, LWP and IWP from MODIS dataset for March 2017 (areas that have no data are shown in white). Anchor figure2_2 figure2_2
3. Application(s)
...
specific assessments
In addition to the extensive product validation (see chapter 2 for results and chapter 2/3 in [D4] for validation methodology) a second assessment is introduced to evaluate the Interim Climate Data Record (ICDR) against the Thematic Climate Data Record (TCDR) in terms of consistency. Since frequent ICDR deliveries make detailed validation not feasible, a consistency check against the deeply validated TCDR is used as an indication of quality. This is done by a comparison of the following two evaluations:
- TCDR against a stable, long-term and independent reference dataset
- ICDR against the same stable, long-term and independent reference dataset
The evaluation method is generated to detect differences in the ICDR performance in a quantitative, binary way with so called Key Performance Indicators. The general method is outlined in [D6] chapter 3. The same difference between TCDR/ICDR and the reference dataset would lead to the conclusion that TCDR and ICDR have the same quality (key performance is "good"). Variations or trends in the differences (TCDR/ICDR against reference) would require a further investigation to analyze the reasons. The key performance would be marked as "bad". The binary decision whether the key performance is good or bad is made in a statistical way by a hypotheses test (binomial test). Based on the TCDR/reference comparison (global means, monthly or daily means) a range is defined with 95% of the differences are within. This range (2.5 and 97.5 percentile) is used for the ICDR/reference comparison to check whether the values are in or out of the range. The results could be the following:
- All or a sufficient high number of ICDR/reference differences lies within the range defined by the TCDR/reference comparison: Key performance of the ICDR is "good"
- A smaller number of ICDR/reference differences is within the pre-defined range: Key performance of the ICDR is "bad"
3.1 Results
The results of the KPI test are summarized in Table 3-1.
Anchor | ||||
---|---|---|---|---|
|
Cloud Fractional Cover | Cloud Top Pressure | Ice Water Path | Liquid Water Path | ||
---|---|---|---|---|---|
Percentiles | p2.5 p97.5 | -1% 3% | -7.9 hPa 6.93 hPa | -16.4 g/m² 11.3 g/m² | -4.65 g/m² 4.48 g/m² |
Sentinel-3A: | |||||
01/2017 - 12/2020 | 36/48 | 08/48 | 40/48 | 18/48 | |
01/2017 - 12/2021 | 12/60 | 41/60 | 20/60 | 39/60 | |
01/2017 - 06/2022 | 12/66 | 41/66 | 26/66 | 42/66 | |
Sentinel-3B: | |||||
10/2018 - 12/2021 | 07/39 | 26/39 | 26/39 | 26/39 | |
10/2018 - 06/2022 | 07/45 | 26/45 | 32/45 | 33/45 | |
Sentinel-3A+B: | |||||
10/2018 - 06/2022 | 02/45 | 42/45 | 06/45 | 27/45 |
Percentiles were calculated based on the comparison of the TCDR using the Advanced Along Track Scanning Radiometer (AATSR) instrument against MODIS Collection 6.1 (Terra satellite only) as reference dataset for the variables Cloud Fractional Cover (CFC), Cloud Top Pressure (CTP), Ice Water Path (IWP) and Liquid Water Path (LWP). Percentiles were based on the time from 2002-2012 with monthly means and applied to the ICDR from 01/2017 (10/2018) to 06/2022 for Sentinel-3A (Sentinel-3B and merged product Sentinel-3A+B) based on measurements of the Sea and Land Surface Temperature Radiometer (SLSTR).
Most of the ICDR months are outside the TCDR-based KPI limits and leading to “bad” KPI tests. Therefore, the ICDR is not stable in relation to the TCDR. This is due to multiple reasons starting with the fact of a five year gap (2012-2016) between TCDR and ICDR. In addition, TCDR and ICDR are based on different instruments with SLSTR on Sentinel-3 and (A)ATSR/ATSR-2 on Envisat/ERS-2, respectively. Differences occur due to a lower bias between ICDR and reference dataset and a subtraction of the monthly means (based on the TCDR) to remove the annual cycle leads to values outside of the KPI range (see method in [D6], chapter 3.2.2). On the other hand, IWP has a higher bias compared with MODIS which can be caused by different cloud classification due to differences in the visible channels between (A)ATSR and SLSTR. Please note that significant changes between 01/2017 - 12/2020 and 01/2017 - 12/2021 are due to bugfixesThis section is not applicable. There are no additional application specific assessments known since the dataset has just been published.
4. Compliance with user requirements
...
- Monthly mean IWP is computed using only daytime measurements.
- Similar limitations as mentioned for Cloud liquid water path (LWP)
Info |
---|
This document has been produced with funding by the European Union in the context of the Copernicus Climate Change Service (C3S).The activities leading to these results have been contracted , operated by the European Centre for Medium-Range Weather Forecasts , operator of C3S on behalf on the European Union (Contribution Agreement signed on 22/07/2021). All information in this document is provided “as is” "as is" and no guarantee of warranty is given that the information is fit for any particular purpose. The users thereof use the information at their sole risk and liability. For the avoidance of all doubt, the European Commission and the European Centre for Medium-Range Weather Forecasts have no liability in respect of this document, which is merely representing the author’s author's view. |
Related articles
Content by Label | ||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|