Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Issued by: SMHI/Karl-Göran Karlsson

Date: 03/06/10/20232024

Ref:  C3S2C3S2_D312a_Lot1.3.1.1-20222023_TRGAD-CLD_v1.1

Official reference number service contract: 2021/C3S2_312a_Lot1_DWD/SC1

Info
titlePlease cite as
Karlsson, K.-G., et al., (20232024): C3S Cloud Properties CDRs releases until March 20232024: Target Requirements and Gap Analysis Document. Copernicus Climate Change Service. Document reference C3S2_D312a_Lot1.3.1.1-20222023_TRGAD-CLD_v1.1. Last accessed on dd/mm/yyyy

...

Expand
titleClick here to expand the history of modifications


Version

Date

Description of modification

Chapters / Sections

V1.0

2830/04/20232024

Original version covering all deliverances deliveries between start of Phase II until March 20232024

All

V1.1

03/06/10/20232024

Document revised following feedback from independent review

All


...

Expand
titleClick here to expand the list of related documents (D1-D14)


Reference ID

Document

D1

[GCOS-154] Systematic Observation Requirements for Satellite-based Products for Climate Supplemental details to the satellite-based component of the Implementation Plan for the Global Observing System for Climate in Support of the UNFCCC, 2011 Update, December 2011. World Meteorological Organization, Geneva, Switzerland. Available from https://library.wmo.int/doc_num.php?explnum_id=3710 .

D2

[GCOS-200] The Global Observing system for climate: Implementation needs, 2016, World Meteorological Organization, Geneva, Switzerland. Available from http://www.wmo.int.

D3

CM SAF CDOP2 Product Requirement Document, SAF/CM/DWD/PRD, v3v4.70

Available upon request from Deutscher Wetterdienst (DWD)

D4

CLARA-A2.1 Product User Manual, CM SAF Cloud, Albedo, Radiation data record, AVHRR-based, Edition 3 (CLARA-A3) Cloud Properties

Code: Cloud products, SAF/CM/DWD/PUM/GACCLARA/CLD, v2.6, 15.05.2020

Available from:

Issue 3.1

https://www.cmsaf.eu/SharedDocs/Literatur/document/

2020

2023/saf_cm_dwd_pum_

gac

clara_cld_

2

3_

6

1_pdf.

html

pdf?__blob=publicationFile

Last accessed on 07/08/2024

D5

Validation Report, CM SAF Cloud,

AVHRR GAC cloud products, Edition 2.1,

Albedo, Radiation data record, AVHRR-based, Edition 3 (CLARA-A3) Cloud Properties

Code: SAF/CM/DWD/VAL/GAC/CLD

/2.6, 15.05.2020

, Issue 3.1

Available from:

https://www.cmsaf.eu/SharedDocs/Literatur/document/

2020

2023/saf_cm_smhi_val_

gac

clara_cld_

2

3_

6

1_pdf.

html

pdf?__blob=publicationFile

Last accessed on 07/08/2024

D6

Algorithm Theoretical

Baseline Document CLARA-A2 Cloud Products;

Basis Document, CM SAF Cloud, Albedo, Radiation data record, AVHRR-based, Edition 3 (CLARA-A3) Cloud Properties

Code: SAF/CM/DWD/ATBD/CLARA/CLD

/2.5, 13.02.2020

, Issue 3.3

Available from:

https://www.cmsaf.eu/SharedDocs/Literatur/document/

2020

2023/saf_cm_dwd_atbd_

gac

clara_cld_

2

3_

5

3_pdf.pdf?__blob=publicationFile

Last accessed on 07/08/2024

D7

Algorithm Theoretical Baseline Basis Document Cloud Mask Products for the Cloud Probability of the NWC/PPS (NWC SAF PPS v2014v2018, v2021); SAFNWC/CDOP3/CMPPS/SMHI/SCI/ATBD/CMA_AVHRR/CloudProbability, Issue: 2.10, 1526.0504.20202021

Available from:

https://www.cmsafnwcsaf.euorg/SharedDocsDownloads/LiteraturPPS/documentv2021/2020/saf_cm_smhi_atbd_gac_cma_avhrr_2_1_pdf.htmlDocuments/NWC-CDOP3-PPS-SMHI-SCI-ATBD-CloudProbability_v2_0.pdf

Last accessed on 29/04/2024

D8

Algorithm Theoretical Baseline Basis Document for Cloud Top Height Products Temperature, Pressure and Height of the NWC/PPS (NWC SAF PPS v2014); SAF/CM/SMHI/ATBD/CTX_AVHRR/2.1, 15.05.2020v2018, v2021, NWC/CDOP3/PPS/SMHI/SCI/ATBD/CTTH, Issue 3.0, 26.04.2021
Available from:

https://www.cmsafnwcsaf.euorg/SharedDocsDownloads/LiteraturPPS/documentv2021/2020/saf_cm_smhi_atbd_gac_ctx_avhrr_2_1_pdf.htmlDocuments/NWC-CDOP3-PPS-SMHI-SCI-ATBD-CTTH_v3_0.pdf

Last accessed on 07/08/2024

D9

Algorithm Theoretical Baseline Basis Document Cloud Physical Products for Cloud Micro Physics of the NWC/PPS (NWC SAF PPS v2014); SAF/CMv2018, v2021), NWC/CDOP3/PPS/SMHI/SCI/ATBD/CPP_AVHRR, 15.05.2020CMIC, Issue 3, Rev. 0, 26.04.2021
Available from:

https://www.cmsafnwcsaf.euorg/SharedDocsDownloads/LiteraturPPS/documentv2021/2020/saf_cm_smhi_atbd_gac_cpp_avhrr_2_1_pdf.htmlDocuments/NWC-CDOP3-PPS-SMHI-SCI-ATBD-CMIC_v3_0.pdf

Last accessed on 07/08/2024

D10

Meirink, J.F. (KNMI) et al (2022) C3S

Service: Key Performance Indicators (KPIs), Copernicus Climate Change Service,

Document ref. C3S_D312b_Lot1.0.4.8_201903_UpdatedKPIs_v1.0

https://confluence.ecmwf.int/x/AM_BEQ

Last accessed on 2107/0208/20232024

D11

ESA Cloud CCI Product User Guide, v.5.1, 16.01.2020.

Available from:

https://climate.esa.int/media/documents/Cloud_Product-User-Guide-PUG_v5.1.pdf

Last accessed on 0507/0408/20232024

D12

ESA Cloud CCI Product Validation and Intercomparison Report, v.6.0, 03.02.2020.

Available from:

https://climate.esa.int/media/documents/Cloud_Product-Validation-and-Intercomparison-Report-PVIR_v6.0.pdf

Last accessed on 0507/0408/20232024

D13

ESA Cloud CCI Algorithm Theoretical Basis Document, v.6.2, 14.10.2019.

Available from:

https://climate.esa.int/media/documents/Cloud_Algorithm-Theoretical-Baseline-Document-ATBD_v6.2.pdf

Last accessed on 0507/0408/20232024

D14

ESA Cloud CCI Algorithm Theoretical Basis Document: Community Cloud retrieval for Climate (CC4Cl), v.6.2, 18.10.2019.

Available from:

https://climate.esa.int/media/documents/Cloud_Algorithm-Theoretical-Baseline-Document-ATBD-CC4CL_v6.2.pdf

Last accessed on 0507/0408/20232024


Acronyms

Expand
titleClick here to expand the list of acronyms


Acronym

Definition

(A)ATSR

(Advanced) Along Track Scanning Radiometer; used here to refer to the ATSR-2 and AATSR instruments collectively

AHI

Advanced Himawari Imager (on Japanese geostationary satellite Himawari)

ANN

Artificial Neural Network

AVHRR

Advanced Very High-Resolution Radiometer

BC-RMSD

Bias corrected RMSD (equal to cRMSD)

BRDF

Bidirectional Reflectance Distribution Function

CA

Cloud Amount

CALIOP

Cloud-Aerosol Lidar with Orthogonal Polarization

CALIPSO

Cloud-Aerosol Lidar and Infrared Pathfinder Satellite Observations

CC4Cl

Community Cloud retrieval for Climate

CCI

Climate Change Initiative (ESA)

CCI+

Follow-on project of ESA’s Climate Change Initiative

CDR

Climate Data Record

CECR

Comprehensive Error Characterisation (ESA Cloud CCI)

CEOS

Committee on Earth Observation Satellites

CER

Cloud Effective Radius

CFC

Cloud Fractional Cover (similar to CA)

CGMS

Coordination Group for Meteorological Satellites

CIMSS

Cooperative Institute for Meteorological Satellite Studies

CLARA-A1

CM SAF Cloud, Albedo and Radiation data record – AVHRR-based, Edition 1

CLARA-A2

CM SAF Cloud, Albedo and Radiation data record – AVHRR-based, Edition 2

CLARA-A3

CM SAF Cloud, Albedo and Radiation data record – AVHRR-based, Edition 3

Cloud_cci

ESA’s Climate Change Initiative on Clouds

Cloud CCI+

Extension of Cloud_cci project

CMAPROB

Probabilistic cloud masking method used in PPS

CM SAF

Satellite Application Facility on Climate Monitoring

COT

Cloud Optical Thickness

cRMSD

Centered RMSD (or BC-RMSD)

CTH

Cloud Top Height

CTP

Cloud Top Pressure

CTT

Cloud Top Temperature

C3S

Copernicus Climate Change Service

DLR

Deutsches Zentrum für Luft- und Raumfahrt (German Aerospace Center)

DWD

Deutscher Wetterdienst (Germany’s National Meteorological Service)

EarthCARE

Earth Clouds, Aerosols and Radiation Explorer (ESA)

ECMWF

European Centre for Medium-range Weather Forecasts

ECV

Essential Climate Variable

ENVISAT

Environmental Satellite (ESA)

EPS-SG

EUMETSAT Polar System - Second Generation

ERA-Interim

A global atmospheric reanalysis produced by the European Centre for Medium‐Range Weather Forecasts

ESA

European Space Agency

EUMETSAT

European Organization for the Exploitation of Meteorological Satellites

FCDR

Fundamental Climate Data Record

FCI

Flexible Combined Imager (EUMETSAT)

FDR

Fundamental Data Record

FIDUCEO

Fidelity and uncertainty in climate data records from Earth Observations

FOV

Field of VeiwView

Fy-3D

Feng-Yun-3D, a Chinese polar orbiting satellite

GAC

Global Area Coverage (AVHRR)

GCOS

Global Climate Observing System

GEO

Group on Earth Observations

GOES

Geosynchronous Operational Environmental Satellite (USA)

GSICS

Global Space-based Inter-Calibration System

HIRS

High-resolution Infrared Radiation Sounder

IFS

Integrated Forecast System (ECMWF)

ISCCP

International Satellite Cloud Climatology Project

IWP

Ice Water Path

JPSS

Joint Polar Satellite System

KNMI

The Meteorological Institute of the Netherlands

KPI

Key Performance Indicator

LEO

Low Earth Orbit

LWP

Liquid Water Path

LTDN

Local Time on Descending Node

LUT

Lookup Table

METEOSAT

Meteorological geostationary satellite (EUMETSAT)

MERIS

Medium Resolution Imaging Spectrometer (on ENVISAT)

MERSI

Medium Resolution Spectral Imager

METimage

Meteorological Imager

Metop

Meteorological Operational Satellite

MODIS

Moderate-resolution Imaging Spectroradiometer

MSG

METEOSAT Second Generation (EUMETSAT)

MSR

Multi Sensor Reanalysis

MTG

METEOSAT Third Generation (EUMETSAT)

NASA

National Aeronautics and Space Administration

NCEI

National Center for Environmental Information (USA)

NEODC

UK National Earth Observation Data Centre

netCDF

Network Common Data Format

NISE

Near-real-time Ice and Snow Extent product

NOAA

National Oceanic and Atmospheric Administration

NWC SAF

Satellite Application Facility on Support to Nowcasting and Very Short-Range Forecasting

NWP

Numerical Weather Prediction

OLCI

Ocean and Land Colour instrument (Sentinel-3 satellite)

ORAC

Optimal Retrieval of Aerosol and Cloud

OSI SAF

Satellite Application Facility on Ocean and Sea Ice

OSCAR

Observing Systems Capability Analysis and Review Tool (WMO)

PATMOS-x

Pathfinder Atmospheres - Extended

PPS

Polar Platform System (cloud processing software package developed in NWC SAF)

RMSD

Root Mean Square Deviation

RTTOV

Radiative Transfer for the Television and Infrared Observation Satellite Operational Vertical Sounder

S3VT

Sentinel-3 Scientific Validation Team

SCM-5

5th project within SCOPE-CM: Advancing the status of the AVHRR FCDR

SCOPE-CM

Sustained, Coordinated Processing of Environmental Satellite Data for Climate Monitoring

SEVIRI

Spinning Enhanced Visible and Infrared Imager (EUMETSAT)

SLSTR

Sea and Land Surface Temperature Radiometer

SMHI

Swedish Meteorological and Hydrological Institute

Suomi-NPP

Suomi National Polar-orbiting Partnership

TCDR

Thematic Climate Data Record

TIROS-N

Television and Infra-Red Observation Satellite – N

TOA

Top of atmosphere

TRGAD

Target Requirements and Gap Analysis Document

USGS

United States Geological Survey

VGAC

VIIRS data resampled to AVHRR GAC resolution

VIIRS

Visible Infrared Imaging Radiometer Suite

WCRP

World Climate Research ProgrammeProgram

WMO

World Meteorological Organization


...

Expand
titleClick here to expand the list of tables

Table 2‑1: GCOS target requirements from GCOS-154 for six Cloud Properties: Cloud Amount (CA), Cloud Top Pressure (CTP), Cloud Top Temperature (CTT), Cloud Optical Depth (COD), Cloud Water Path (CWP) and Cloud Effective Radius (CRE)

. From reference D1 (page 29). Observe that GCOS parameter names differ slightly from what otherwise is used in this document (for example, CA=CFC, COD=COT and CRE=CER)

Table 2‑2: Key Performance Indicators (KPIs) or target requirements (i.e., fulfilled requirements for CLARA-

A2.1 in

A3 in the CM SAF project) for the cloud products CFC, CTP, LWP and IWP (monthly means) of interest for C3S_312b_Lot1

. More details are given in reference D5 (Table 1.1 to Table 1.3).

Table 2‑3: KPI percentiles (2.5 % and 97.5 %) requirements for the CLARA-

A2.1

A3 CFC, CTP, LWP and IWP product differences against MODIS C6

.1.

Table 2‑4: GCOS-154 requirements for CFC, CTP, LWP and IWP compared to CLARA-

A2.1 requirements.

A3 requirements (bias in absolute numbers)

Table 2‑5: Key Performance Indicators (KPIs) for the CFC, CTP, CTT, COT, CER, LWP and IWP products (monthly means) of Cloud_cci v3.0 of interest for C3S_312b_Lot1

.

Table

2-6

2‑6: KPIs to be applied to corresponding SLSTR ICDR products in Table

2-4

2‑5, based on comparison of the Cloud_cci v3.0 TCDR against MODIS Collection 6.1

.

Table 3‑1: Spectral channels of the Advanced Very High Resolution Radiometer (AVHRR)

. The three different versions of the instrument are described as well as their corresponding satellites. Notice that channel 3A was only used continuously on NOAA-17 and on the Metop satellites. For the other satellites with AVHRR/3 it was used only for shorter periods. (Table taken from reference document D4 but extended with information on satellites Tiros-N and Metop-C).

Table 3‑2: Channel 3A and 3B activity for the AVHRR/3 instruments during daytime

. Notice that the given time periods show the availability in the CLARA TCDR v2.0 and not the true lifetime of the individual sensor/satellite. The table is taken from reference document D4 (slightly modified with respect to end of data record).

Table 3‑3: Spectral channels of the Visible/Infrared Imager Radiometer Suite (VIIRS) sensor carried by the Suomi-NPP and JPSS satellites

Table 3‑3: Spectral channels of the Visible/Infrared Imager Radiometer Suite (VIIRS) sensor carried by the Suomi-NPP and JPSS satellites. (From https://weather.msfc.nasa.gov/sport/jpsspg/viirs.html).

Table 3‑4: Spectral channels of the Meteorological Imager (METimage) sensor to be carried on the EPS-SG satellites

. (From DLR METimage presentation at 3rd Post-EPS User Workshop 29 September 2011)

Table 3‑5: Next generation polar orbiting meteorological satellites in low earth orbit (LEO) for the US/European Joint Polar Satellite System

. Satellites are listed in chronological order based on true and planned launch dates. All information is taken from the WMO OSCAR site (https://www.wmo-sat.info/oscar/satellites/) in April 2022.

List of figures

Expand
titleClick here to expand the list of figures

Figure 3‑1: Local solar times at equator observations for all satellites from

NOAA

TIROS-

7

N to NOAA-19 and Metop A/B

. Shown are all data that were used for the CLARA-A2.1 processing. Notice that the figure shows both ascending (northbound) and descending (southbound) daytime equator crossing times. Each satellite has also another equator crossing occurring 12 hours later (at night or in the evening). (Figure taken from reference document D4).

Figure 3‑2: Overview of all NOAA satellites (except NOAA-19 launched in 2009) carrying the AVHRR instrument. Notice that the figure only shows expected operational life and not the true operation periods for individual AVHRR instruments. Figure 3-3 gives a more complete picture with all AVHRR-carrying satellites (including EUMETSAT Metop satellites). (From https://www.nesdis.noaa.gov/content/noaa-15-makes-100-thousandth-orbit).

Figure 3‑3: Overview of all satellites carrying the AVHRR instrument on individual satellites from Tiros-N in 1978 until 2022. Vertical lines mark new launches or end of operations for individual AVHRRs (From EUMETSAT, 2023)

...

/C

Figure 3‑2: Overview of all satellites carrying the AVHRR instrument on individual satellites from TIROS-N in 1978 until 2022

Figure 3‑3: Local solar times at equator observations for all satellites with the AVHRR and VIIRS sensors (compare with Figure 3‑1)

Anchor
general_definitions
general_definitions
General definitions

Climate data records

Climate data compilations from observations are most often referred to as Climate Data Records (CDRs). However, the data records from satellites may consist of different types of quantities, from original radiances to derived products. Radiance data of climate quality are defined as Fundamental Climate Data records (FCDRs) while data records consisting of satellite-derived geophysical products are defined as Thematic Climate Data Records (TCDRs). In the ideal case the TCDRs should be derived by methods using FCDRs as input. However, if standards for the used radiances have not fulfilled the strict requirements for being classified as FCDRs, these radiances may be denoted Fundamental Data Records (FDRs). Notice that TCDRs can currently be based on either FCDRs or FDRs.

A special case of TCDRs are data records produced with short latency (e.g., shortly after the end of a month). These are called Interim Climate Data Records (ICDRs). The word Interim means that the data record has a higher uncertainty than the original TCDR since it has not been possible to use exactly the same input data as for the TCDR due to the short latency. Interim also means that a user may have to wait for the next edition of the TCDR to get a fully consistent and homogenous climate data record that includes data from the period with ICDR data. Normally ICDRs behave very similar to TCDRs but continuous monitoring of their quality is recommended.

Notice that since ICDRs are continuous extensions of the TCDR they are also delivered at subsequent times in separate batches (numbered 1,2,3..etc) where each one covers a certain time period (e.g. a number of months). Thus, when formally describing the full ICDR in the text (i.e., using the name specified in the delivery

Climate data records

Climate data compilations from observations are most often referred to as Climate Data Records (CDRs). However, the data records from satellites may consist of different types of quantities, from original radiances to derived products. Radiance data of climate quality are defined as Fundamental Climate Data records (FCDRs) while data records consisting of satellite-derived geophysical products are defined as Thematic Climate Data Records (TCDRs). In the ideal case the TCDRs should be derived by methods using FCDRs as input. However, if standards for the used radiances have not fulfilled the strict requirements for being classified as FCDRs, these radiances may be denoted Fundamental Data Records (FDRs). Notice that TCDRs can currently be based on either FCDRs or FDRs.

A special case of TCDRs are data records produced with short latency (e.g., shortly after the end of a month). These are called Interim Climate Data Records (ICDRs). The word Interim means that the data record has a higher uncertainty than the original TCDR since it has not been possible to use exactly the same input data as for the TCDR due to the short latency. Interim also means that a user may have to wait for the next edition of the TCDR to get a fully consistent and homogenous climate data record that includes data from the period with ICDR data. Normally ICDRs behave very similar to TCDRs but continuous monitoring of their quality is recommended.

Notice that since ICDRs are continuous extensions of the TCDR they are also delivered at subsequent times in separate batches (numbered 1,2,3..etc) where each one covers a certain time period (e.g. a number of months). Thus, when formally describing the full ICDR in the text (i.e., using the name specified in the delivery list), the ICDR version number is given but the batch number is written in generic form using letter x, for example ICDR v1.x. This is just to indicate that the batch number is only describing a temporal increment of the product and not any change of the product.

...

Depending on the data record producer, different product requirements may be applied and they are used to evaluate validation results. An often-used way to handle this is to define several levels of requirements where each level is linked to specific needs or priorities. A three-level approach like the following is rather common:

RequirementDescription
Threshold requirement

A product should at least fulfill this level to be considered useful at all. Sometimes the term ‘Breakthrough” is used instead.

Target requirement

This is the main quality goal for a product. It should reach this level based on the current knowledge on what is reasonable to achieve.

Optimal requirement

This is a level where a product is considered to perform much better than expected given the current knowledge.


Satellite product levels

Satellite-based products are often described as belonging to the following condensed description of processing levels, each one with different complexity and information content:

LevelDescription
Level-0

Raw data coming directly from satellite sensors, often described as sensor counts.

Level-1

Data being enhanced with information on calibration and geolocation.

Three sub-levels are often referred to:

Level-1a: Data with attached calibration and geolocation information

Level-1b: Data with applied calibration and attached geolocation information

Level-1c: Data with applied calibration and additional layers of geolocation, satellite viewing and solar angle information

Level-2

Derived geophysical variables at the same resolution and location as L1 source data.

An often-used Level-2 variety is the following:

Level-2b: Globally resampled images, two per day per satellite, describing both ascending (passing equator from south) and descending (passing equator from north) nodes. Resampling is based on the principle that the value for the pixel with the lowest satellite zenith angle is chosen in case two or several swaths are overlapping.

Level-3

Gridded data with results accumulated over time (e.g., monthly means).

A more comprehensive definition of all processing levels is given here: https://www.earthdata.nasa.gov/engage/open-data-services-and-software/data-information-policy/data-levels.

...

  1. The Cloud Properties TCDR AVHRR CLARA v2v3.0 product (known as CLARA-A2.1A3) and its associated ICDR v2v3.x,

  2. The Cloud PropertiesTCDR Properties TCDR ESA AATSR v3.0 product and its associated SLSTR ICDR v3.1.1 and v4.0.

...

The products associated with the Cloud Properties TCDR AVHRR CLARA v2v3.0 (covering the period 19821979-20182020) are described together with their target requirements. For the four cloud products (i) cloud fractional cover (CFC), (ii) cloud top pressure (CTP), (iii) liquid water path (LWP) and (iv) ice water path (IWP) the mean error requirements are 5 2 %, 50 27 hPa, 10 -8 gm-2 and 20 -0.4 gm-2, respectively. Corresponding stability requirements are 2 -1.27 % per decade, 20 -4.7 hPa per decade, 3 -6.9 gm-2 per decade and 6 -0.8 gm-2 per decade, respectively.

Key Performance Indices (KPIs) for ICDR products to be produced for the extension of the TCDR (so far covering the period January 2019 2021 until June 2022October 2023) have been defined using MODIS Level 3 products as reference observations. The KPI test is based on a binomial test against low (2.5%) and high (97.5%) percentiles of the MODIS-CLARA difference distribution. The low and high percentiles are given in the following table:

Variable

KPI: lower percentile

(2.5 %)


KPI: higher percentile

(97.5 %)

CFC

-0.

718

575 %

0.

576

571 %

CTP

-

7

8.

036

534 hPa

4

7.

353

501 hPa

LWP

-0.

0021

0036 g/

m² 

0.

0022

0041 g/m2

IWP

-0.

0031

0048 g/m²

0.

0021

0030 g/m²

An extensive description of past, current and future availability of data from the Advanced Very High-Resolution Radiometer (AVHRR) is given. In addition, future prospects of utilizing utilising AVHRR-heritage spectral channel data from new imaging sensors on new satellites are described. It is concluded that the AVHRR-based observations series, based on one morning and one afternoon orbit constellation, can be prolonged to reach at least 60 years if adding AVHRR-heritage information. However, for this to become effective, efforts are needed to harmonize and homogenize observations between true AVHRR data and AVHRR-heritage data. This concerns both calibration aspects and spatial resolution aspects. Regarding the development of retrieval methods, it is noted that the access to high quality cloud observations from active sensors (especially CALIPSO-CALIOP data) has played an important role in advancing both retrieval methods and methods for uncertainty estimations in recent years. The access to the high-quality reference data has been especially important for the development of Bayesian and artificial neural network (ANN) based retrievals. A future continuation of active observations from space is judged as crucial for further development of retrieval methods based on AVHRR-heritage data. Dissemination of data from CALIPSO-CALIOP was discontinued in 2022 and near-future access to data from the new EarthCARE mission (scheduled for launch in 2024) will be essential for further development and validation activities.

The cloud products provided by the Cloud Properties TCDR ESA AATSR v3.0 are described, along with their target requirements. In addition to the Cloud Properties provided by the AVHRR CLARA v2.0 TCDR (CFC, CTP, LWP and IWP), the Cloud_cci TCDR also provides (i)cloud top temperature (CTT), (ii) cloud optical thickness (COT) and (iii) cloud effective radius (CER). The mean error requirements for these variables are 5K, 10% and 10%, respectively, with the corresponding stability requirements being 1K, 2% and 2% per decade.

The data record of the (A)ATSR sensors, running from 1995-2012, and the ICDR from the SLSTR sensor (covering 2017-2022) are discussed. Options for filling the four-year gap between the end of the AATSR data record and the beginning of the SLSTR record are also provided. A key advantage of the (A)ATSR and SLSTR data (compared to AVHRR) is that the data are highly stable, both in terms of satellite orbital parameters and instrument calibration, at the cost of temporal coverage and, in the case of (A)ATSR, instrument swath width. With the introduction of the second operational SLSTR sensor on Sentinel-3B in 2018, it is possible for SLSTR to provide nearly global coverage twice a day by combining data from both platforms. For C3S, such a combined product has been provided as the v4.0 data version.

1.  Product Product description

Cloud products estimated from satellite sensors describe one component of the atmospheric part of the global water cycle where the other parts are Water Vapour and Precipitation products. To get a full picture of the global water cycle, components from the Earth´s surface have to be added (e.g., evaporation, ice and snow accumulation/melting and runoff estimates).

Two cloud data records, both based on visible and infrared imagery, are delivered to C3S. One is based on data from the Advanced Very High-Resolution Radiometer (AVHRR). AVHRR-data forms the longest available time series of meteorological imagery with observations starting in 1978. Here, a data record with data from 1982 1979 and onwards is delivered. The second data record is based on data from the Along Track Scanning Radiometer (ATSR) and its successor AATSR (i.e., Advanced ATSR). The used observation series started in 1995 and ended in 2012. An extension of the data record, with similar observations from the Sea and Land Surface Temperature Radiometer (SLSTR) starting in 2017, is also provided. 

1.1 Cloud Properties TCDR AVHRR CLARA

...

v3.0 + ICDR

...

v3.x

This section describes cloud products of the CLARA-A data record (CM SAF CM SAF CLoud, Albedo and Radiation data record – AVHRR-based) and its ICDR extension. This data record is produced by the EUMETSAT Satellite Application Facility on Climate Monitoring (CM SAFCM SAF) and exists currently in three four versions: CLARA-A1, CLARA-A2 and , CLARA-A2.1 and CLARA-A3. The reason for the difference in version numbering between the C3S and CM SAF CM SAF projects is that C3S never included the CLARA-A1 data record. The version described here as version 23.0 is the extended version of the second third and latest edition of this dataset, denoted CLARA-A2.1 A3 by the CM SAFCM SAF. In the remainder of this document we will use the shorter notation CLARA-A2.1 A3 to refer to this data record.

The CLARA-A2.1 A3 CDR is based on Advanced Very High-Resolution Radiometer (AVHRR) observations onboard the NOAA and EUMETSAT Metop satellites. Observations are available in reduced spatial resolution globally (approximately 5 km, denoted Global Area Coverage or GAC) and the final CDR is compiled in a regular global grid with 0.25̊ (~25 km) resolution. The time period covered by the TCDR is from January 1982 1979 to December 20182020. ICDR production started in January 2019 April 2023 and products have been delivered covering the period from January 2019 2021 until June 2022October 2023. Products were based on an updated AVHRR Fundamental Climate Data Record (FCDR) where corrections for the visible channels (Heidinger, 2018) originated from a method described by Heidinger et al. (2010).

In addition to being based on AVHRR radiances, the CLARA-A2.1 A3 cloud retrieval algorithms make use of the following auxiliary datasets:

Due to the unavailability ICDR production uses practically the same input data but due to latency reasons preliminary versions of ERA-5 data (denoted ERA-5T) and slightly different OSI SAF products (OSI-401b and OSI 401d) are used. In addition, since relying on visible calibration from vicarious calibration efforts, regular updates are used, when available.

The CLARA-A3 Interim reanalysis data from 2019 onwards, the ICDR production (starting in January 2019) is thereafter utilizing ECMWF Integrated Forecast System (IFS) analyses as auxiliary Numerical Weather Prediction (NWP) input. The alternative to use ERA-5 instead of ERA-Interim as input is not possible with the required latency. The CLARA-A2.1 cloud TCDR and ICDR cover in total six different cloud products (see Karlsson et al., 20172023) but for the C3S_312b_Lot1 and C3S2_312a_Lot1 projects exclusively four products were selected. These are described in the following sub-sections. The basic processing chain and detailed algorithm descriptions are given in reference documents D6-D9.

1.1.1 Cloud Fractional Cover (CFC)

...

This product is calculated using the NWC SAF PPS (Polar Platform System) version 2014 2018 (incl. patch 2) cloud mask probability algorithm (see see https://www.nwcsaf.org/ and http://nwcsaf.smhi.se/ for details on the NWC-SAF project). The algorithm (detailed by Dybbroe et al., 2005) is based on a multi-spectral thresholding technique applied to every pixel of the satellite sceneis based on Naïve Bayesian theory, where cloud probabilities for individual image feature values are sequentially multiplied, followed by scaling with the likelihood of simultaneous image feature value occurrence, yielding a total cloud probability. The implemented method is called CMAPROB and is described in detail by Karlsson et al. (2020). A special feature of CMAPROB is that probabilities vary with surface type, following earlier findings by Karlsson and Håkansson (2018) based on comparisons with CALIPSO-CALIOP data (Winker, 2016). The binary cloud mask is based on the thresholding of the cloud probability at the 50 % probability level. A detailed description is given in D8[D7].

1.1.2 Cloud Top Pressure (CTP)

The Cloud Top Pressure (CTP) is one of the variables that forms the CLARA-A2.1 A3 Cloud Top Level product suite (although all are derived using the same basic algorithms):1.

  1. Cloud Top Temperature (CTT), expressed in Kelvin;

...

  1. Cloud Top Height (CTH), expressed as altitude over ground topography (m);

...

  1. Cloud Top Pressure (CTP), expressed in pressure co-ordinates (hPa).

The product selected for C3S_312b_Lot1 and C3S2_312a_Lot1 is the Cloud Top Pressure product (CTP). However, all three types of the Cloud Top products are accessible (i.e., all three products are included in the original brokered product files) and they are all mutually compatible.

The CTP product is derived using two approaches, one for opaque and one for fractional and semitransparent clouds, and they are applied to all cloudy pixels as identified by the PPS cloud mask product. The opaque algorithm matches measured 11 μm radiances with cloud free and cloudy TOA 11 μm radiances simulated from reference atmospheric profiles. The semi-transparent algorithm estimates mean CTT from two-dimensional histograms of 11 minus 12 μm brightness temperatures versus 11 μm brightness temperatures by use of a curve-fitting technique. As a final step, CTT is converted to CTP using reference atmospheric profiles.based on an artificial neural network (ANN, of the type multilayer perceptron) trained offline with collocations of passive imager measurements from AVHRR and CALIPSO-CALIOP (Winker et al., 2016) cloud top pressure observations (as outlined by Håkansson et al., 2018). The 11 µm channel is used together with the 12 µm channel for AVHRR/2 and AVHRR/3. However, when CTO is based on AVHRR/1, where the 12 µm channel is missing, the 3.7 µm channel is used instead. The CTP product also includes error estimates retrieved using Quantile Regression Neural Networks (see Pfreundschuh et al., 2018) for the 16th and 84th percentiles.

The algorithm is part of NWC SAF PPS v2018, patch 2Both schemes are part of NWC SAF PPS 2014 patch 1. Details can be found in [D8].

Level 3 CTP products are basically derived in the same way as for the CFC product, i.e., averaging of daily Level 2b CTP products on a global 0.25° latitude/longitude grid. However, the standard linear averaging is complemented with an alternative logarithmical averaging as a consequence of the non-linear decrease of pressure with geometrical height. The logarithmic averaging gives a CTP product that is more consistent with the geometrical CTH product.

...

The liquid water path (LWP, units kg m-2) is derived exclusively for daytime conditions and exclusively for cloudy pixels having the liquid phase near the cloud top as deduced from the CLARA-A2.1 A3 cloud phase product. The product is calculated from the estimated cloud optical thickness (τ) and the particle effective radius (re) following the concept introduced by Nakajima and King (1990). It is based on the principle that the reflectance of clouds at a (for cloud particles) non-absorbing wavelength in the visible region (e.g., 0.6 or 0.8 μm) is strongly related to τ and has little dependence on re, whereas the reflectance of clouds at an absorbing wavelength in the shortwave-infrared region (e.g., 1.6 or 3.7 μm) is strongly dependent on effective radius. Lookup Tables (LUTs) of top-of-atmosphere reflectances at the respective pair of channels as a function of viewing and illumination geometry, τ and re are defined from radiative transfer calculations, and used in the retrieval process.

Liquid water path is then computed from the retrieved retrieved τ and re values by (Stephens, 1978):

Mathinline
LWP = \frac{2}{3} \rho_l \tau r_e

, in which 

Mathinline
\rho_l

2/3 ρl τ re, in which ρl represents the density of liquid water (1000 kg m-3).

More details on the retrieval scheme can be found in [D9].

Level 3 LWP products are averaged linearly onto the 0.25° latitude/longitude grid from Level 2b products for all cloudy pixels with the liquid phase. Results are also calculated for all-sky conditions, i.e. by including cloud-free and ice-cloud pixels as zeroes in the averaging.

...

The ice water path (IWP, units kg m-2) is derived in exactly the same way as LWP but now for cloudy pixels with the frozen (ice) phase near the cloud top, and using using

Mathinline
IWP = \frac{2}{3} \rho_j \tau r_e

...

represents the density of ice (930 kg m-3). Also here, Level 3 results are available as both cloudy-sky and all-sky averages. Details on the retrieval scheme can be found in [D9].

1.2 Cloud Properties TCDR ESA AATSR v3.0+ ICDR ESA SLSTR v3.1.1+v4.0

...

Cloud_cci provides three values defining the Cloud Top, all of which are different parameters derived from the same retrieval:1.

  1. Cloud Top Temperature (CTT), expressed in Kelvin;

...

  1. Cloud Top Height (CTH), expressed as altitude over ground topography (km);

...

  1. Cloud Top Pressure (CTP), expressed in pressure co-ordinates (hPa).

The products selected for C3S_312b_Lot1 are the Cloud Top Pressure product (CTP) and Cloud Top Temperature (CTT). However, all three types of the Cloud Top product are accessible (i.e., all three products are included in the original brokered product files) and they are all mutually compatible.

...

The relationship between the observed TOA radiances and the cloud effective radius is strongly impacted by the phase – liquid or ice water – of the observed cloud (as well as the ice crystal habitat in the case of ice cloud). Cloud phase was determined through the application of the Pavolonis Pavalonis cloud typing algorithm (Pavalonis et al., 2005), with additional tests on retrieved CTT. In the case of liquid water, the cloud particle size distribution is modelled as a modified gamma distribution, whereas ice clouds are modelled using the “general habit mixture” ice scattering properties provided by Baum et al. (2010, 2014).

...

The cloud water path (CWP, units kg m-2) is a derived product, determined from the retrieved COT and CER during the day, and from the cloud effective emissivity (which represents equivalent microphysical information to optical depth and effective radius in a single parameter, which can be used when the information content of the measurements is too low to derive two separate parameters). The calculation of CWP follows the approach of Han et al. (1994):

Mathinlinemathdisplay
CWP = \frac{4}{3}\frac{\tau_{c}r_{e}\rho}{Q_{e}}  (Eq 1)

...

The CWP is calculated from level-2 products (i.e. at 1x1 km resolution). When computing the daily and monthly products provided via the CDS, CWP is split into separate averages for Liquid Water Path (LWP) and Ice Water Path (IWP), the sum of which can be considered the total CWP value.

Anchor
section2
section2
2. User Requirements

This section describes the requirements which have been set to be achieved by the described products. Requirements can be set at different levels (as explained in the section with General definitions) but here we will focus on what is called the Target Requirements. These requirements define the main goals for data producers which have to be fulfilled by their products. Requirements are specified by the use of various accuracy parameters which are also listed in the section with General definitions. Observe that for brokered products the target requirements are set to the achieved validation results since these products are not developed and tested within the C3S project.

Concerning products to be used in climate monitoring, requirements for what should be achievable through Earth Observation systems are generally defined by the World Meteorological Organisation (WMO) Global Climate Observation System (GCOS) expert panel. However, these requirements are generally oriented towards the capability and resolution of climate models with a rather course spatial resolution while many products listed here focus more on the monitoring of local and regional scale conditions. Also, they are often not attainable using existing or historical observing systems. Thus, GCOS requirements are not always identical to the requirements listed here since also other user groups other than the climate modelling community have contributed in setting the requirements. However, the relation to GCOS requirements are is discussed below for each individual product.

2.1 Cloud Properties TCDR AVHRR CLARA

...

v3.0 + ICDR

...

v3.x

2.1.1 Introduction to user requirements

...

This document mainly focuses on the user requirements for the climate research and climate modelling community, although some relevant aspects from other user groups are also covered.
To understand requirements better we shortly summarize the major points of interest for each cloud product described above, considering that the current spread in climate model predictions is to a large extent explained by differences in the description of clouds, or more clearly the cloud feedback processes:

...

  • In order to understand a cloud’s impact on radiation processes, it is necessary to describe how a cloud is changing radiation conditions through scattering processes when radiation is passing through the cloud. Thus, the COT parameter can be used to define the extinction of radiation through scattering processes. COT is also closely linked to cloud emissivity which describes a cloud’s ability to emit radiation. Consequently, COT is an important parameter when estimating the greenhouse effect of clouds (as mentioned above for CTP).

...

To evaluate the quality of a product, certain validation procedures and rules have to be followed. A good overview is given in the introduction section in the Report on Updated KPIs (D10)[D8]. For the CLARA-A2.1 A3 cloud products, we have used the Bias, the centered root-mean-squared deviation (cRMSD) and stability as defined previously (General definitions) as quality descriptors.

...

In the CM SAF project, requirements for all products were defined in three levels (threshold, target and optimal) reflecting both the maturity of the retrieval and the estimated optimal capability of the measurement or sensor. For C3S_312b_Lot1 and C3S2_312a_Lot1 we will only discuss one of these levels: the target requirement.

An important aspect is whether requirements are defined for Level 2 (instantaneous) products or for Level 3 (time-averaged) products. Validation of the Level 2 product must be considered as the most accurate evaluation method provided that measurement and reference observations can be adequately matched in space and time, and that a statistically significant time series of evaluated measurements can be compiled. Level 3 products are generally defined at a spatially reduced resolution (through spatial averaging) which means that especially the variability as defined by the precision parameter (cRMSD) is reduced. On the other hand, several additional error sources may alter the Bias parameter, such as insufficient spatial (i.e., where measurements are lacking or only intermittently available) and temporal (i.e., missing days or parts of the daily observations) sampling. A very important error source is also that Level 3 products from reference observations will almost never be possible to compile in exactly the same way as the product to be evaluated. All these additional potential error sources must be taken into account considered when defining target requirements for Level 3 products.

...

The most relevant requirements for the climate research community are specified by United Nations-ratified program the Global Climate Observation System (in specific GCOS requirement documents. For planning and definition of CLARA-A3 products, GCOS-154, see reference [D1]) was the relevant GCOS document. Later versions (e.g. GCOS-200, [D2], were released too late to be considered. Table 2-1 below lists the current GCOS target requirements from GCOS-154 for a set of Cloud Properties. The relevant entries in this table for the current C3S_312b_Lot1 and C3S2currentC3S2_312a_Lot1 deliverables are Cloud Amount (CA) (matching CFC), CTP (matching CTP) and Cloud Water Path (CWP) (matching both LWP and IWP). These requirements have been determined by assuming a cloud feedback similar to a radiative forcing of about 0.3 Wm-2, which is roughly 20 % of the current greenhouse gas forcing.

Anchor
table2_1
table2_1
Table 2‑1: GCOS target requirements from GCOS-154 for six Cloud Properties: Cloud Amount (CA), Cloud Top Pressure (CTP), Cloud Top Temperature (CTT), Cloud Optical Depth (COD), Cloud Water Path (CWP) and Cloud Effective Radius (CRE). From reference D1 (Requirements are found on page 29 )in [D1]. Observe that GCOS parameter names differ slightly from what otherwise is used in this document (for example, CA=CFC, COD=COT and CRE=CER)

...

Some minor changes to the requirements in Table 2-1 have been introduced recently (in 2016) in GCOS-200 (reference [D2 (], page 279):

  • The stability requirement for Cloud Amount (CA) and Cloud Top Temperature (CTT) was changed to 0.01 per decade and 0.25 K per decade, respectively.

  • The accuracy (Bias) requirement for Cloud Effective Radius (CRE) was changed to 1 µm.

  • The stability for Cloud Effective Radius (CRE) was changed to 1 µm per decade.

However, these recent changes have not been taken into account considered when developing the CLARA-A2.1 A3 data record (i.e., the original CLARA-A2 A3 CDR was completed before 2016) but in 2021-2022 after a long preparation process). However, they will be relevant for upcoming new CLARA-A editions. Notice also that the requirements for LWP and IWP in [D3] are defined in absolute numbers and not in relative numbers as in [D1].

2.1.4 Summary of target requirements (KPIs)

The CLARA-A2.1 A3 cloud products are brokered from the CM SAF project, and consequently cannot be altered in C3S_312b_Lot1 and C3S2_312a_Lot1. Therefore, the current target requirements on the key performance of the data set, measured within C3S by the so-called Key Performance Indicators (KPIs), are defined as the requirements results achieved (with original requirements described in [D3]) in previous CLARA-A2.1 A3 validation activities in CM SAF. These values are listed in Table 2-2. For the CLARA-A3 evaluation and validation efforts, various datasets were used as reference. The KPIs in Table 2-2 reflect the evaluation results based on comparisons with the reference dataset [D5] that was considered most appropriate for the particular variable.

Anchor
table2_2
table2_2
Table 2‑2: Key Performance Indicators (KPIs) or target requirements (i.e., fulfilled requirements for CLARA-A2.1 A3 in the CM SAF project) for the cloud products CFC, CTP, LWP and IWP (monthly means) of interest for C3S_312b_Lot1. More details are given in reference D5 [D4] (Table 1.1 to Table 1.3).

.

Variable

KPI: accuracy (Bias)

Fulfilled by CLARA-

A2.1

A3 CDR

KPI: decadal stability

Fulfilled by CLARA-

A2.1

A3 CDR

CFC

5 %

2 % 

-1.27

2

% /decade

CTP

50

27 hPa

20 hPa

-4.7 hPa/decade

LWP

10

-5.7 g/m²

3

1.8 g/m² decade

IWP

20

-0.4 g/m²

6

-0.8 g/m² decade


For the evaluation of the ICDR, corresponding products from the MODIS instrument (MODIS Collection 6.1) are used as a reference. The distribution of the de-seasonalised differences between MODIS products and the CLARA.A2.1 A3 TCDR has been compiled and the corresponding 2.5 and 97.5 percentile differences are given in Table 2-3.

Anchor
table2_3
table2_3
Table 2‑3: KPI percentiles (2.5 % and 97.5 %) requirements for the CLARA-A2.1 A3 CFC, CTP, LWP and IWP product differences against MODIS C6.1..

Variable

KPI: lower percentile

(2.5 %)


KPI: higher percentile

(97.5 %)

CFC

-0.

718 %

575 % 

0.

576

571 %

CTP

-

7

8.

036

534 hPa

4

7.

353

501 hPa

LWP

-0.

0021

003586 g/m²

0.

0022

004053 g/m2

IWP

-0.

0031

004792 g/m²

0.

0021

002958 g/m²

These percentiles are used to check, by means of a binomial test at 5 % significance level, whether the corresponding ICDR differences are consistent with the TCDR differences or not. Further details on these tests are found in the Report on Updated KPIs (D10).

...

Table 2-4 shows how the CLARA-A2.1 A3 TCDR requirements compare to GCOS requirements. The achieved CLARA-A2.1 A3 results in Table 2-4 are estimated at a product grid resolution of 0.25° which is approximately twice as fine as the GCOS-referred values. The reason for this is explained by requirements of higher spatial resolutions from other user categories than the climate research community (e.g., regional NWP and climate modelers, and the solar energy community). Despite the finer resolution, GCOS requirements are met for both CFC and CTP (at least when considering the slightly relaxed requirements for semi-transparent clouds). For LWP and IWP (both referring to the CWP requirement in Table 2-1 ) compliance is also good even if requirements are not expressed in the same way by GCOS as by CM SAF. Details Further details of the validation results can be found in Karlsson et al. (20172023) and in [D5]. Regarding the GCOS requirement on the temporal resolution (interpreted as the basic required observation frequency for calculation of useful Level-3 products), the CLARA-A2.1 A3 TCDR can only fulfil this requirement over the entire observation period at moderate to high latitudes. The tropical region will generally only be covered every 4th to 8th hour during the first two decades while later the observations frequency has increased with a maximum frequency better than 3 hours occurring around in the period 2009-2014. After 2014, orbital drift effects slowly decrease the observation frequency again.

Anchor
table2_4
table2_4
Table 2‑4: GCOS-154 requirements for CFC, CTP, LWP and IWP compared to CLARA-A2.1 requirements.A3 requirements (bias in absolute numbers)

Requirements

GCOS (Target)

CLARA-A2.1 TCDR + ICDR v2.x

Spatial resolution

50 km

25 km

Temporal resolution

3-hourly

Monthly

Accuracy:

CFC

CTP

LWP

IWP


1-5 %

15-50 hPa

25 %

25 %

5


2 %

50

27 hPa

10

8 gm-2

20

0.4 g m-2

Stability:

CFC

CTP

LWP

IWP


0.3-3 %/decade

3-15 hPa/decade

5 %/decade

5 %/decade

2


1.27 %/decade

20

4.7 hPa/decade

3

6.9 g/m2/decade

6

0.8 g/m2/decade

2.1.6 Data format and content issues

The CLARA-A2.1 A3 cloud property products are defined using standard data formats (netCDF) and map projections (regular latitude/longitude grids). Meta data definitions follow the Climate & Forecast conventions (http://cfconventions.org/).

2.2 Cloud properties TCDR ESA AATSR v3.0 + ICDR ESA SLSTR v3.1.1+v4.0

As the Cloud_cci products cover the same ECVs as the CLARA-A2.1 A3 CDR, the requirements are very similar. In particular, the users of the data can be considered to be the same and the same GCOS requirements specified in Table 2-1 apply. Due to the consistent overpass time of (A)ATSR and SLSTR, the Cloud_cci can be considered to be a more stable and consistent record of Cloud Properties, and it contains more properties

Footnote

Although it should be noted that CLARA-

A2.1

A3 based COT and CER estimates are available from the CMSAF; they just aren’t included in the dataset brokered to C3S.

, than CLARA-A2.1A3. However, the Cloud_cci CDR is considerably shorter in time than the CLARA-A2.1 A3 record and there is a four-year gap between the end of the TCDR and the SLSTR ICDR. Also, (A)ATSR provides sparser spatial coverage (due to its relatively narrow swath) compared to AVHRR. SLSTR has a wider swath than AATSR (although still narrower than AVHRR) and, from 2018 onwards, two SLSTR instruments flying with interleaved orbits provide global twice-daily coverage (with 10 am and 10 pm local-time overpasses).

The additional parameters provided in the Cloud_cci CDR over the CLARA-A2.1 A3 products provide some additional scope for the application of the data. Particular points of interest include:

...

2.2.1 Summary of target requirements (KPIs)

Similar to the CLARA-A2.1 productA3 TCDR, Cloud_cci v3 is a brokered product from ESA’s CCI programmeprogram, and cannot be substantively changed for C3S_312b_Lot1 and C3S2_312a_Lot1. Thus, the KPIs for the Cloud_cci data are taken from the data quality and verification analysis performed within the Cloud_cci project itself. The resulting KPIs are summarized in Table 2-5.

Anchor
table2_5
table2_5
Table 2‑5: Key Performance Indicators (KPIs) for the CFC, CTP, CTT, COT, CER, LWP and IWP products (monthly means) of Cloud_cci v3.0 of interest for C3S_312b_Lot1.

Variable

KPI: accuracy (Bias)

Fulfilled by Cloud_cci TCDR

KPI: decadal stability

Fulfilled by Cloud_cci TCDR

CFC

8.09 %

-0.35 % /decade

CTP

-25.52 hPa

3.99 hPa/decade

CTT

No individual requirement. Compliance of CTP, LWP and IWP considered.

COT

2.4 (liquid), 0.58 (ice)

0.01 (liquid), -0.02 (ice) /decade

CER

-1.8(liquid), -12.5 (ice)

Mathinline
\mu m


-0.09 (liquid),-0.12 (ice)

Mathinline
\mu m

/decade

LWP

-17.29 g/m²

0.99 g/m² decade

IWP

-28.77 g/m²

-2.27 g/m² decade

In order to monitor the performance of the SLSTR extension ICDR, new KPI values were also generated from comparisons of the Cloud Properties TCDR ESA AATSR to the NASA MODIS Cloud Properties product. These comparisons are represented as the 2.5 and 97.5 percentiles of the distribution of differences between (A)ATSR monthly-mean values and the corresponding MODIS values (corrected for the mean seasonal cycle), and are

...

summarized in Table 2-6.

Anchor
table2_6
table2_6
Table 2-6: KPIs to be applied to corresponding SLSTR ICDR products in Table 2-45 , based on comparison of the Cloud_cci v3.0 TCDR against MODIS Collection 6.1.

Variable

KPI:

2.5thpercentile


KPI:

97.5th percentile

CFC

-1.3 %

3.1 %

CTP

-7.9 hPa

6.9 hPa/decade

CTT


No individual requirement. Compliance of CTP, LWP and IWP considered.


COT

CER

LWP

-4.65 g/m²

4.48 g/m² decade

IWP

-16.4 g/m²

11.3 g/m² decade

2.2.2 Data format and content issues

The Cloud_cci v3 Cloud Property products are defined using standard data formats (netCDF) and map projections (regular latitude/longitude grids). Meta data definitions follow the Climate & Forecast conventions (http://cfconventions.org/).

Anchor
section3
section3
3. Gap Analysis

3.1 Description of past, current and future satellite coverage

3.1.1 Cloud Properties TCDR AVHRR CLARA

...

v3.0 + ICDR

...

v3.x

The CLARA-A2.1 A3 TCDR is based on radiances measured by the AVHRR sensor. This sensor is carried by satellites travelling in sun-synchronous polar orbits with daytime equator crossing times either in the morning or in the afternoon. With the nominal constellation of at least two satellites (i.e., one morning and one afternoon satellite) this gives four observations per day at the equator while the observation frequency increases at higher latitudes (at maximum 28 observations per day at the poles) due to increasing overlaps between the satellite swaths. The sensor is basically a five-channel instrument but it has been available in three different versions with slightly modified channel compositions as illustrated in Table 3-1. This table covers all historically available satellites and AVHRR versions, i.e., not only those covered by CLARA TCDR v2.0. The missing satellites for the TCDR AVHRR CLARA v2.0 are Tiros-N, NOAA-6, NOAA-8, NOAA-10 and Metop-Cversions where all versions have now been used in the CLARA-A3 TCDR. Note that the third version of the instrument (AVHRR/3) is capable of switching between two different channels in the short-wave infrared spectral region (i.e. channels 3A and 3B). The periods when switching was actually carried out for the data used in TCDR AVHRR CLARA v2v3.0 are presented in in Table 3-2. Note that data from Metop-B was not used for the TCDR in the years 2016-2018 due to errors in the calibration of visible channels. However, both Metop-B and Metop-C data are used in the ICDR data record after getting access to updated calibration data.

Anchor
table3_1
table3_1
Table 3‑1: Spectral channels of the Advanced Very High-Resolution Radiometer (AVHRR). The three different versions of the instrument are described as well as their corresponding satellites. Notice that channel 3A was only used continuously on NOAA-17 and on the Metop satellites. For the other satellites with AVHRR/3 it was used only for shorter periods. (Table taken from reference document D4 but extended with information on satellites Tiros-N and Metop-C[D4]).

Channel
Number

Wavelength
(micrometers)
AVHRR/1
Tiros-N, NOAA-6,8,10

Wavelength
(micrometers)
AVHRR/2
NOAA-7,9,11,12,14

Wavelength
(micrometers)
AVHRR/3
NOAA-15,16,17,18
NOAA-19, Metop-A, Metop-B, Metop-C

1

0.58-0.68

0.58-0.68

0.58-0.68

2

0.725-1.10

0.725-1.10

0.725-1.10

3A

-

-

1.58-1.64

3B

3.55-3.93

3.55-3.93

3.55-3.93

4

10.50-11.50

10.50-11.50

10.50-11.50

5

Channel 4 repeated

11.50-12.50

11.50-12.50

Anchor
table3_2
table3_2
Table 3‑2: Channel 3A and 3B activity for the AVHRR/3 instruments during daytime. Notice that the given time periods show the availability in for the CLARA TCDR v2v3.0 and not the true lifetime of the individual sensor/satellite. The table is taken from reference document [D4] (slightly modified with respect to end of data record).

Satellite

Channel 3A active

Channel 3B active

NOAA-15


06/1998 – 12/

2015

2020

NOAA-16

10/2000 – 04/2003

05/2003 – 12/2011

NOAA-17

07/2002 – 02/2010


NOAA-18


09/2005 – 12/

2018

2020

NOAA-19

03/2009 – 05/2009

06/2009 – 12/

2018

2020

Metop-A

09/2007 – 12/

2018

2020


Metop-B

01/2013 – 12

&2015

/2020


Metop-C

07/2019 – 12/2020


In the following text we will refer to some figures prepared for the description of the complete CLARA-A2.1 A3 data record from the CM SAF project (described in [D4). It is identical to the TCDR AVHRR CLARA v2.0 except that it also includes data for the first six months of 2019]).

Figure 3-1 shows an overview of all NOAA satellites with AVHRR measurements which were used for the compilation of the CLARA-

...

A3 TCDR.

...

This figure also shows how observation times have changed due to various degrees of orbital drift for the individual satellites. Another prominent feature seen in Figure 3-1 is the temporal observation inhomogeneity. For example, initially only one (afternoon) satellite was used while during the last

...

15 years of the series 4-5 individual satellites could be used in parallel.

...

Image Added

Anchor
figure3_1
figure3_1
Figure 3‑1: Local solar times at equator observations for all satellites from NOAATIROS-7 N to NOAA-19 and Metop A/B/C. Shown are all data that were used for the CLARA-A2.1 A3 processing. Notice that the figure shows both ascending (northbound) and descending (southbound) daytime equator crossing times. Each satellite has also another equator crossing occurring 12 hours later (at night or in the evening). (Figure taken from reference document [D4]).

The lack of morning satellite data in the beginning of the period is explained by the retrieval algorithm requirement to have all 5 spectral AVHRR bands available. AVHRR/1 has only 4 spectral bands, thus lacking full coverage with simultaneously operating satellites in both morning and afternoon orbits is only partly fulfilled in the first decade of observations, Also, observe that AVHRR/1 for TIROS-N, NOAA-6, NOAA-8 and NOAA-10 in the beginning of the satellite series has only 4 spectral bands (Table 3-1). The lack of the split-window channel at 12 µm being crucial is critical for some aspects, e.g., thin Cirrus detection and accurate cloud top height determination. Updated retrieval algorithms based on AVHRR/1 data can only partly compensate for this loss of basic information.

The variable coverage of The variable coverage of AVHRR observations over the CLARA-A2.1 A3 time period is problematic in the sense that it violates the requirement of having as homogeneous observations as possible when compiling CDRs. However, it can be noticed that the inhomogeneity concerns especially the satellites in the morning orbit (i.e., with equator crossing times in the interval 7-10 LST in Figure 3-1) while observations from the afternoon orbit have more adequate coverage over the entire time period. Thus, meaningful studies of e.g. climate trends are still possible if restricting the study to the use of data from afternoon satellites (as demonstrated by Karlsson and Devasthale, 2018 and by Stengel et al., 2020 and Devasthale and Karlsson, 2023).

If looking a bit closer on the actual data coverage, it is clear that some satellites have experienced technical problems of varying kinds (e.g., scan motor problems, instrument noise in the AVHRR 3.7 micron µm channel, etc.) which have led to data losses or partially corrupt scenes. Noise problems in the 3.7 micron µm channel affected all satellites carrying AVHRR/1 and AVHRR/2 sensors. Problems were normally weak or moderate but some satellites experienced particularly high and severe noise levels during some periods of their lifetime (e.g., TIROS-N, NOAA-6, NOAA-7, NOAA-9, NOAA-10 and NOAA-12). The periods of high noise levels lead typically to overestimated cloud amounts from most of the used cloud screening methods. For example, for the CLARA-A2.1 A3 predecessor CLARA-A1, it contributed to creating a large negative trend in cloud cover because of overestimated cloud amounts for the early satellites in the series of used satellites (Sun et al., 2015). A noise filtering procedure was introduced for CLARA-A2.1 (Karlsson et al., 2017) which reduced the impact of excessive 3.7 micron µm noise. This filtering method is also used for CLARA-A3.

Serious gaps in data coverage have also affected the AVHRR observation series. The largest gap occurred between September 1994 (loss of NOAA-11) and April 1995 (launch of NOAA-14). A few other gaps also exist but normally covering less than a month´s worth of data. Problems with partially corrupt scenes have also temporarily led to a decrease of the frequency of available observations. Corrupt scenes (or scenes with a sub-set of corrupt scan lines) are normally associated with scan motor problems and may occur for all satellites, and especially towards the end of the AVHRR instrument life time. A particularly problematic period occurred in the period 2000-2003 when several satellites had problems with corrupt scan lines from AVHRR. This was challenging for various parameter retrieval schemes since not all corrupt scan lines in these scenes were correctly labelled as corrupt in the original Level 1b files. Therefore, in the CLARA-A2.1 case, large efforts were undertaken to secure blacklisting of corrupt lines while at the same time using as much as possible of remaining non-affected scan lines.

The extension of a TCDR (in this case extending CLARA-A2 with three additional years 2016-2018) an AVHRR-based TCDR and the introduction of ICDR production after the end of the official length of the TCDR period (2019+) are not trivial to carry out since some of the necessary input data may be lacking or be restricted during the extension period. This is one of the reasons why this type of extension is called “Intermediate” (i.e., being or Intermediate Climate Data record (ICDR). These products are produced with some restrictions compared to the original TCDR) while awaiting the next full reprocessing effort (i.e., CLARA-A3).for the next TCDR edition.

For the CLARA-A2.1 A3 case, a particular problem occurred regarding the access to a stable calibration of visible channels which takes into account considers the degradation with time of the sensor response. The original CLARA-A2 A3 TCDR relied upon the updated results provided by NOAA following the original approach by Heidinger et (Heidinger, 2018), using a slightly revised method compared to the original approach by Heidinger et al. (2010). Unfortunately, the production of latest access to these updated calibration results from this particular method ceased after 2016for CLARA-A3 is from 2017. It means that all results from 2017 and onwards are based on extrapolated calibration data which are not adjusted after access to more recent calibration reference data in the years after 2020. Since these calibration corrections have non-linear terms which are not very well determined for the youngest satellites, deviations may grow in an unrealistic way with time. This was especially the case for the visible AVHRR radiances from the Metop-B satellite and it was clear that this affected the derived CLARA products in a negative way. Consequently, it was decided to exclude Metop-B data from CLARA-A2. 1 for the extension period 2016-2018. However, updates of the Metop-B and Metop-C calibration have been received recently (20202023) which means that data from these satellites is now included in the ICDR data records. However, it shows that problems still remain for TCDR and ICDR results in the period 2017-2022 and these have to be taken care of in the next edition of CLARA.

The CLARA-A3 TCDR observation period can be extended both back in time (until 1978, see Figure 3-2 and Figure 3-3) and forward in time taking into account considering satellites that are in operation from 2019 2021 until now (presently only covered by ICDR products, see Figure 3-2).

Such an extension is planned in the CM SAF project and this new CLARA version will be denoted CLARA-A3.5. The new version will tentatively cover the time period 19781979-20202024, i.e. almost 42 years (in particular, covering the new reference climatology period 1991-2020). It will partly solve the observation inhomogeneity seen for CLARA-A2 in the beginning of the period by adding more satellites with the AVHRR/1 sensor. However, this will also require handling of new potential inhomogeneities due to the inclusion of a more restricted AVHRR instrument. Retrievals based on AVHRR/1 are expected to lead to slightly decreased cloud detection capabilities (especially concerning thin Cirrus clouds) and to some degraded cloud top height estimation accuracy (again, especially for high thin clouds).

As already mentioned, the backward extension will solve the lack of morning satellite data in the CLARA-A2.1 dataset in the period 1982-1991. The missing data will then be provided by the morning satellites NOAA-6, NOAA-8 and NOAA-10. Regarding the earliest period 1978-1981, a full coverage with both morning and afternoon satellites is unfortunately not possible. Afternoon observations will start in October 1978 with the Tiros-N satellite which is then later replaced by NOAA-7 in June 1981. However, a data gap of several months (March until May) will exist in 1981. Similarly, morning observations will start in June 1979 with the NOAA-6 satellite and it is later replaced by NOAA-8 in March 1983. It means that for the first half year with afternoon observations (i.e., October 1978 until May 1979) morning observations will be missing.

Image Removed

...

46 years.

Image Added

Anchor
figure3_2
figure3_2
Figure 3‑2: Overview of all satellites carrying the AVHRR instrument on individual satellites from TIROS-N in 1978 until 2022. Vertical lines mark new launches or end of operations for individual AVHRRs (From EUMETSAT, 2023)

However, CLARA-A3 is most likely the last CLARA TCDR edition based exclusively on data from the AVHRR instrument. The very last satellite carrying the AVHRR sensor is Metop-C which was launched on 7th of November 2018. With a nominal expected lifetime of approximately 5 years it means that AVHRR data beyond 2025 (or maybe 2030 considering the extended life of most recent AVHRRs) will most likely not be available anymore. The last satellite carrying AVHRR in an afternoon orbit is NOAA-19. This satellite has now been operational for more than 15 years and it is clearly approaching the end of its lifetime. A particular problem with the still available NOAA satellites in afternoon orbit carrying the AVHRR sensor is that orbital drift has already affected them seriously. In fact, for the last few years of the CLARA-A3 TCDR (at least years 2019 and 2020) these observations are better characterized as morning observations than afternoon observations as a consequence of the orbital drift. This is another reason for putting a definite end of exclusively AVHRR-based TCDRs to the year 2020 (see further discussion below).

The AVHRR-based CDRs covering more than four decades are unique and precious datasets from the climate monitoring point of view in that they represent the longest available multispectral imaging observation record from space. Possibilities to extend the observation series even further by use of data from other sensors would therefore be extremely valuable. The prospects here are good considering that the next generation polar-orbiting meteorological satellites will carry sensors with AVHRR-heritage spectral bands as a subset of their full range of spectral bands. Some of these satellites are already in orbit. The most important sensors in this respect are the Visible/Infrared Imager Radiometer Suite (VIIRS) and the Meteorological Imager (METimage) sensors. Both sensors’ spectral channels are described in Table 3-3 and Table 3-4, respectively. We note in Table 3-3 that heritage AVHRR channels are channels M5, M7, M10, M12, M15 and M16 for VIIRS. For METimage in Table 3-4, AVHRR-heritage channels are available at wavelengths 670 nm, 865 nm, 1630 nm, 3740 nm, 10790 nm and 12020 nm.

Table 3-5 lists the current and planned satellites (including their expected lifetimes) carrying the VIIRS and METimage sensors. The VIIRS sensor is carried on the next generation US polar orbiting meteorological satellites following the NOAA-19 satellite. The first satellite with VIIRS was Suomi-NPP which was launched already in 2011. Suomi-NPP was a precursor to the US part of the operational Joint Polar Satellite System program (JPSS). The first JPSS satellite (JPSS-1) was launched in 2017 and it was later renamed to NOAA-20. It was followed by NOAA-21 launched in 2022. The US part of JPPS deals with satellites in the afternoon orbit. The second leg, i.e. the morning satellites, will be provided by EUMETSAT with the upcoming EPS-SG satellites. These satellites will carry the METimage sensor.

It should be noted that VIIRS observations in a stable afternoon orbit have been available since 2011. This means that VIIRS data can not only be used to extend CLARA-A3 beyond 2020 but it can also be used to replace data from NOAA-18 and NOAA-19 in the period 2011-2020 when the latter satellites were exposed to serious orbital drift. In other words, VIIRS data can be used to restore the nominal two orbit constellation of observations from AVHRR during the last decade of CLARA-A3 coverage. It means that VIIRS data can not only extend CLARA-A3 results but also improve CLARA-A3 results for the last decade of its temporal coverage. This aspect is an essential part in the planning of CLARA-A3.5. CLARA-A3.5 will then add results derived from the AVHRR-heritage channels of VIIRS from Suomi-NPP, NOAA-20 and NOAA-21. CLARA-A3.5 is scheduled for release in 2026. Figure 3-3 shows equator crossing times for all satellites with both the AVHRR sensor (compare with Figure 3-1) and the VIIRS sensor.

Anchor
table3_3
table3_3
Table 3‑3: Spectral channels of the Visible/Infrared Imager Radiometer Suite (VIIRS) sensor carried by the Suomi-NPP and JPSS satellites. (Fromhttps://weather.msfc.nasa.gov/sport/jpsspg/viirs.html).

Image Added

Anchor
table3_4
table3_4
Table 3‑4Spectral channels of the Meteorological Imager (METimage) sensor to be carried on the EPS-SG satellites. (From DLR METimage presentation at 3rd Post-EPS User Workshop 29 September 2011)

Image Added

Anchor
table3_5
table3_5
Table 3‑5Next generation polar orbiting meteorological satellites in low earth orbit (LEO) for the US/European Joint Polar Satellite System. Satellites are listed in chronological order based on true and planned launch dates. Notice that NOAA-20 and JPSS-2 (NOAA-21) have been launched according to plan. All information is taken from the WMO OSCAR site (https://www.wmo-sat.info/oscar/satellites/) in April 2024 

Satellite

Provider

Start

Expected

end of life

Orbit
(daytime eq. crossing time)

Imaging sensor

NOAA-20

NOAA

2017

2027

13:25 pm

VIIRS

NOAA-21

NOAA

2022

2030

13:30 pm

VIIRS

Metop-SG-A1

EUMETSAT

2025

2033

09:30 am

MetImage

JPSS-3

NOAA

2032

2039

13:30 pm

VIIRS

Metop-SG-A2

EUMETSAT

2032

2040

09:30 am

MetImage

JPSS-4

NOAA

2027

2034

13:30 pm

VIIRS

Metop-SG-A3

EUMETSAT

2039

2047

09:30 am

MetImage

Table 3-5 shows that AVHRR-heritage information from one morning and one afternoon orbit will be provided for at least 20 more years from present time (i.e., nominally until 2045). Consequently, it means that CLARA-type CDRs will be possible to compile over a time period exceeding 60 years from the initial launch year of 1978 for the first AVHRR instrument.

It is finally worth mentioning that the Chinese Feng-Yun 3 satellites FY-3D to FY-3H carry the 25-channel VIS-IR spectrometer MERSI-2, which also includes all AVHRR heritage channels (see https://space.oscar.wmo.int/satelliteprogrammes/view/fy_3. Partly, these satellites fly in similar orbits to the JPSS and EPS-SG series. Thus, in case problems are encountered with near future satellite coverage of the satellites in the US/European Joint Polar Satellite System, data from the Feng-Yun satellites can be used as a backup solution provided that calibration homogenization can be achieved. However, it should be noted that the upcoming FY-3E and FY-3H satellites will be operated in an early-morning orbit with an equatorial overpass time of 6:00 am.

3.1.2 Cloud Properties TCDR ESA AATSR v3.0 + ICDR ESA SLSTR v3.1.1+v4.0

The Cloud_cci v3.0 is based on radiances provided by the ATSR series of sensors. These instruments flew on sun-synchronous polar orbiting satellites with daytime equatorial crossing times in the mid-morning: 10:30 Local Time on Descending Node (LTDN) for ATSR-2 and 10:00 LTDN for ENVISAT, with both satellites sharing the same ground track. There were 14.3 orbits per day, meaning 28 equatorial overpasses per-day, with measurements covering a total of 18% of equatorial circumference of the Earth (with equally spaced 512 km swaths). The observation frequency increased at higher latitudes (with a maximum of 14 observations per day at the poles) due to increasing overlaps between the satellite swaths. Both sensors provided the same seven channels (and used the same conical dual-viewing geometry), but not all channels were provided at all times, or at full digitization rate, from ATSR-2, due to limitations of the data bandwidth provided by the ERS-2 platform. Over ocean regions, ATRS visible channels were often only provided in a 256 pixel “narrow-swath” mode. The channels provided by both instruments were centered at 0.55, 0.67, 0.87, 1.6, 3.7, 10.8, 12.0 

Mathinline
\mu m

and the filter band passes were very similar between instruments. Despite the low-data rate modes of ATSR-2, the combination of the very similar instrument specifications, very close orbital parameters and the lack of any significant orbital drift in the ERS-2 and ENVISAT satellites mean that ATSR-2 and AATSR provide a highly consistent data record, especially when compared to that provided by the AVHRR record used by the CLARA-A3 TCDR (although AVHRR provides a much longer data record).

The current TCDR from Cloud_cci v3.0 begins with the launch of ERS-2 in mid-1995 and continues until the failure of ENVISAT in April 2012. Due to instrument problems, there is a six-month data gap in the ATSR-2 record from January to June 1996.

There is an overlap of 1 year of data between the two platforms, between mid-2002 (when ENVISAT was launched) and mid-2003 (when the onboard data storage on ERS-2 failed). There is additional ATSR-2 data available up-to 2009, but this is not global as data could only be collected when the satellite was within line-of-sight with a ground receiving station, and has not been included in the TCDR. There is some scope to push the coverage of the ATSR cloud record back to 1991, by using the ATSR-1 instrument (onboard ERS-1), which also flew in a similar orbit to its successors. However, ATSR-1 lacked the shortwave channels (apart from the 1.6 

Mathinline
\mu m

channel), which would reduce the information available to daylight retrievals and would represent a significant inhomogeneity in the TCDR.

The extension of the ATSR TCDR relies on the SLSTR sensors onboard the Sentinel-3 platform. SLSTR represents a significant upgrade over (A)ATSR, providing a wider swath, two satellites with interleaved orbit swaths, additional channels and the data availability security of an operational system. The Sentinel-3s have a very similar orbit to ENVISAT and the ERS satellites, with a sun-synchronous orbit with an LTDN of 10:00, and 14.3 orbits per-day. However, there is slightly longer than 4-year gap between the end of the (A)ATSR record and the first SLSTR data. There are several options available to fill this gap, as ORAC can be applied to most radiometers with similar channels to those provided by ATSR. Indeed, cloud CDRs of ORAC applied to both MODIS and AVHRR already exist, having been produced in the Cloud_cci program. Furthermore, the CLARA-A3 product also discussed in this document also provides a comparable product which fills the (A)ATSR-SLSTR data gap.

It should also be noted that ORAC could be applied to the VIIRS and METimage instruments described above, which could compliment the SLSTR ICDR.

Anchor
section3_2
section3_2
3.2 Development of processing algorithms

3.2.1 Cloud Properties TCDR AVHRR CLARA v2.0 + ICDR v3.x

3.2.1.1 General aspects common to all selected CLARA products

A fundamental aspect is that TCDRs based on AVHRR data must be based on an accurate AVHRR Fundamental Climate Data Record (FCDR), i.e., quality checked radiances corrected for degradations and with proper inter-calibration between individual satellites. The work with FCDRs is an ever-ongoing task (e.g., new data added to the series must be analysed, inter-calibrated and appropriately corrected for possible degradations).

Another factor that affects all CLARA products (as well as all other available AVHRR-based CDRs) is the inhomogeneity of the observational record, i.e., mainly the different observation frequency in the beginning and in the end of the observational record. This is further enhanced by the fact that some satellites exhibit large orbital drift during their lifetime. In the worst case, these two factors may lead to artificial trends in the CDR as discussed by Karlsson et al. (2018). A way to mitigate these problems is to add AVHRR-heritage data from other sensors to the used observational record (as mentioned in section 3.1.1). For the extension of the CDR beyond the AVHRR era, adaptations have also to be done to cope with the AVHRR-heritage channels on post-AVHRR sensors.

The two mentioned aspects are by far the most important ones for assuring a high quality of an AVHRR-based TCDR. The following sub-sections will provide some more details on these aspects. There are also issues (limitations) with the current retrieval algorithms but these will be discussed more in detail later in sections 3.4 and 3.5.

3.2.1.1.1 Maintaining development of the AVHRR Fundamental Climate Data Record

The AVHRR FCDR used as a basis for CLARA-A3 processing was only inter-calibrated and corrected for sensor degradation for the visible channel 1, the near-infrared channel 2 and the short-wave infrared channel 3a (see Table 3-1). This was based on an extension of the method (Heidinger, 2018) originally proposed by Heidinger et al. (2010). Infrared radiances were still based on the nominal operational NOAA calibration method utilising onboard black body reference sensors. Note that the calibrated AVHRR reflectances and brightness temperatures, used as a basis for the CLARA-A3 TCDR processing, were also compiled as a stand-alone dataset in a joint CM SAF/EUMETSAT effort. This dataset, provided by EUMETSAT is denoted the AVHRR FDR (Fundamental Data Record) with the following dataset doi number: 10.15770/EUM_SEC_CLM_0060 (EUMETSAT,2023a, 2023b).

Continued coordination of AVHRR FCDR developments

...

CLARA-A3 will most likely be the last TCDR based exclusively on data from the AVHRR instrument. The very last satellite carrying the AVHRR sensor is Metop-C which was launched on 7th of November 2018. With a nominal expected lifetime of approximately 5 years it means that AVHRR data beyond 2025 (or maybe 2030 considering the extended life of most recent AVHRRs) will most likely not be available anymore. Of importance here is also that AVHRR observations from a satellite in afternoon orbit will cease in 2022 (according to NOAA). The last satellite carrying AVHRR in an afternoon orbit is NOAA-19. This satellite has now been operational for more than 12 years and it is thus approaching the end of its lifetime. A particular problem with the still available NOAA satellites in afternoon orbit carrying the AVHRR sensor is that orbital drift has already affected them seriously. In fact, for the last few years of the planned CLARA-A3 TCDR (at least years 2019 and 2020) these observations are better characterized as morning observations than afternoon observations as a consequence of the orbital drift. This is another reason for putting a definite end of exclusively AVHRR-based TCDRs to the year 2020.

The AVHRR-based CDRs covering more than four decades are unique and precious datasets from the climate monitoring point of view in that they represent the longest available multispectral imaging observation record from space. Possibilities to extend the observation series even further by use of data from other sensors would therefore be extremely valuable. The prospects here are good considering that the next generation polar-orbiting meteorological satellites will carry sensors with AVHRR-heritage spectral bands as a subset of their full range of spectral bands. Some of these satellites are already in orbit. The most important sensors in this respect are the Visible/Infrared Imager Radiometer Suite (VIIRS) and the Meteorological Imager (METimage) sensors. Both sensors’ spectral channels are described in Table 3-3 and Table 3-4, respectively. We note in Table 3-3 that heritage AVHRR channels are channels M5, M7, M10, M12, M15 and M16 for VIIRS. For METimage in Table 3-4, AVHRR-heritage channels are available at wavelengths 670 nm, 865 nm, 1630 nm, 3740 nm, 10790 nm and 12020 nm. Similar information about METimage is given by the following link: https://www.eumetsat.int/eps-sg-metimage.

Table 3-5 lists the current and planned satellites (including their expected lifetimes) carrying the VIIRS and METimage sensors. The VIIRS sensor is carried on the next generation US polar orbiting meteorological satellites following the NOAA-19 satellite. The first satellite with VIIRS was Suomi-NPP which was launched in 2011. Suomi-NPP was a precursor to the US part of the operational Joint Polar Satellite System program (JPSS). The first JPSS satellite (JPSS-1) was launched in 2017 and it was later renamed to NOAA-20. The US part of JPPS deals with satellites in the afternoon orbit. The second leg, i.e. the morning satellites, will be provided by EUMETSAT with the upcoming EPS-SG satellites. These satellites will carry the METimage sensor.

It should be noted that VIIRS observations in a stable afternoon orbit have been available since 2011. This means that VIIRS data can not only be used to extend CLARA-A3 beyond 2020 but it can also be used to replace data from NOAA-18 and NOAA-19 in the period 2011-2020 when the latter satellites were exposed to serious orbital drift. In other words, VIIRS data can be used to restore the nominal two orbit constellation of observations from AVHRR during the last decade of CLARA-A3 coverage. It means that VIIRS data can not only extend CLARA-A3 results but also improve CLARA-A3 results for the last decade of its temporal coverage. This possibility has been taken up in the planning of a further extension, named CLARA-A3.5, of the CLARA data record in the next CM SAF project phase covering the years 2022-2027. CLARA-A3.5 will then add results derived from the AVHRR-heritage channels of VIIRS from Suomi-NPP and NOAA-20. CLARA-A3.5 is scheduled for release in 2026.

...

Image Removed

...

Image Removed

...

Satellite

...

Provider

...

Start

...

Expected

end of life

...

Orbit
(daytime eq. crossing time)

...

Imaging sensor

...

NOAA-20

...

NOAA

...

2017

...

2024

...

1:30 pm

...

VIIRS

...

JPSS-2

...

NOAA

...

2022

...

2029

...

1:30 pm

...

VIIRS

...

Metop-SG-A1

...

EUMETSAT

...

2024

...

2031

...

9:30 am

...

MetImage

...

JPSS-3

...

NOAA

...

2027

...

2034

...

1:30 pm

...

VIIRS

...

Metop-SG-A2

...

EUMETSAT

...

2031

...

2038

...

9:30 am

...

MetImage

...

JPSS-4

...

NOAA

...

2032

...

2039

...

1:30 pm

...

VIIRS

...

Metop-SG-A3

...

EUMETSAT

...

2038

...

2045

...

9:30 am

...

MetImage

Table 3-5 shows that AVHRR-heritage information from one morning and one afternoon orbit will be provided for at least 20 more years from present time (i.e., nominally until 2045. Consequently, it means that CLARA-type CDRs will be possible to compile over a time period exceeding 60 years from the initial launch year of 1978 for the first AVHRR instrument.

It is finally worth mentioning that the Chinese Feng-Yun 3 satellites FY-3D to FY-3H carry the 25-channel VIS-IR spectrometer MERSI-2, which also includes all AVHRR heritage channels (see https://www.wmo-sat.info/oscar/instruments/view/279). Partly, these satellites fly in similar orbits to the JPSS and EPS-SG series. Thus, in case problems are encountered with near future satellite coverage of the satellites in the US/European Joint Polar Satellite System, data from the Feng-Yun satellites can be used as a backup solution provided that calibration homogenization can be achieved. However, it should be noted that the upcoming FY-3E and FY-3H satellites will be operated in an early-morning orbit with an equatorial overpass time of 6:00 am.

3.1.2 Cloud Properties TCDR ESA AATSR v3.0 + ICDR ESA SLSTR v3.1.1+v4.0

The Cloud_cci v3.0 is based on radiances provided by the ATSR series of sensors. These instruments flew on sun-synchronous polar orbiting satellites with daytime equatorial crossing times in the mid-morning: 10:30 Local Time on Descending Node (LTDN) for ATSR-2 and 10:00 LTDN for ENVISAT, with both satellites sharing the same ground track. There were 14.3 orbits per day, meaning 28 equatorial overpasses per-day, with measurements covering a total of 18% of equatorial circumference of the Earth (with equally spaced 512 km swaths). The observation frequency increased at higher latitudes (with a maximum of 14 observations per day at the poles) due to increasing overlaps between the satellite swaths. Both sensors provided the same seven channels (and used the same conical dual-viewing geometry), but not all channels were provided at all times, or at full digitization rate, from ATSR-2, due to limitations of the data bandwidth provided by the ERS-2 platform. Over ocean regions, ATRS visible channels were often only provided in a 256 pixel “narrow-swath” mode. The channels provided by both instruments were centred at 0.55, 0.67, 0.87, 1.6, 3.7, 10.8, 12.0

Mathinline
\mu m

and the filter band passes were very similar between instruments. Despite the low-data rate modes of ATSR-2, the combination of the very similar instrument specifications, very close orbital parameters and the lack of any significant orbital drift in the ERS-2 and ENVISAT satellites mean that ATSR-2 and AATSR provide a highly consistent data record, especially when compared to that provided by the AVHRR record used by the CLARA-A2.1 TCDR (although AVHRR provides a much longer data record).

The current TCDR from Cloud_cci v3.0 begins with the launch of ERS-2 in mid-1995 and continues until the failure of ENVISAT in April 2012. Due to instrument problems, there is a six-month data gap in the ATSR-2 record from January to June 1996.

There is an overlap of 1 year of data between the two platforms, between mid-2002 (when ENVISAT was launched) and mid-2003 (when the onboard data storage on ERS-2 failed). There is additional ATSR-2 data available up-to 2009, but this is not global as data could only be collected when the satellite was within line-of-sight with a ground receiving station, and has not been included in the TCDR. There is some scope to push the coverage of the ATSR cloud record back to 1991, by using the ATSR-1 instrument (onboard ERS-1), which also flew in a similar orbit to its successors. However, ATSR-1 lacked the shortwave channels (apart from the 1.6

Mathinline
\mu

m channel), which would reduce the information available to daylight retrievals and would represent a significant inhomogeneity in the TCDR.

The extension of the ATSR TCDR relies on the SLSTR sensors onboard the Sentinel-3 platform. SLSTR represents a significant upgrade over (A)ATSR, providing a wider swath, two satellites with interleaved orbit swaths, additional channels and the data availability security of an operational system. The Sentinel-3s have a very similar orbit to ENVISAT and the ERS satellites, with a sun-synchronous orbit with an LTDN of 10:00, and 14.3 orbits per-day. However, there is slightly longer than 4-year gap between the end of the (A)ATSR record and the first SLSTR data. There are several options available to fill this gap, as ORAC can be applied to most radiometers with similar channels to those provided by ATSR. Indeed, cloud CDRs of ORAC applied to both MODIS and AVHRR already exist, having been produced in the Cloud_cci program. Furthermore, the CLARA-A2.1 product also discussed in this document also provides a comparable product which fills the (A)ATSR-SLSTR data gap.

It should also be noted that ORAC could be applied to the VIIRS and METimage instruments described above, which could compliment the SLSTR ICDR.

...

3.2.1 Cloud Properties TCDR AVHRR CLARA v2.0 + ICDR v2.x

3.2.1.1 General aspects common to all selected CLARA products

A fundamental aspect is that TCDRs based on AVHRR data must be based on an accurate AVHRR Fundamental Climate Data Record (FCDR), i.e., quality checked radiances corrected for degradations and with proper inter-calibration between individual satellites. The work with FCDRs is an ever-ongoing task (e.g., new data added to the series must be analysed, inter-calibrated and appropriately corrected for possible degradations).

Another factor that affects all CLARA products (as well as all other available AVHRR-based CDRs) is the inhomogeneity of the observational record, i.e., mainly the different observation frequency in the beginning and in the end of the observational record. Orbital drift effects also contribute but the main problem is the lack of morning observations in the current CLARA-A2.1 CDR in the initial part of the covered period. In the worst case, this may lead to artificial trends in the CDR as discussed by Karlsson et al. (2018). A way to mitigate these problems is to add AVHRR/1 observations to the used observational record (as mentioned in the previous section). For the extension of the CDR beyond the AVHRR era, adaptations have also to be done to cope with the AVHRR-heritage channels on post-AVHRR sensors.

The two mentioned aspects are by far the most important ones for assuring a high quality of an AVHRR-based TCDR. The following sub-sections will provide some more details on these aspects. There are also issues (limitations) with the current retrieval algorithms but these will be discussed more in detail later in sections 3.4 and 3.5.

3.2.1.1.1 Maintaining development of the AVHRR Fundamental Climate Data Record

The AVHRR FCDR used as a basis for CLARA-A2.1 processing was only inter-calibrated and corrected for sensor degradation for the visible channel 1, the near-infrared channel 2 and the short-wave infrared channel 3a (see Table 3-1). This was based on an extension of the method originally proposed by Heidinger et al. (2010). Infrared radiances were still based on the nominal operational NOAA calibration method utilizing onboard black body reference sensors.

In order to coordinate development and to improve progress in calibration methodologies, a special project (SCM-5: Advancing the AVHRR FCDR) was defined in 2014 within the framework of the SCOPE-CM (Sustained and COordinated Processing of Environmental satellite data for Climate Monitoring) activity - https://www.scope-cm.org/). SCOPE-CM is a high-level project network with links to entities like WMO, WCRP, GCOS, CGMS, CEOS and GEO. SCM-5 involved scientists from NOAA and NASA, and also from the CM SAF and ESA Cloud_cci projects. It was also linked to the EU-project FIDUCEO (Fidelity and uncertainty in climate data records from Earth Observations - https://research.reading.ac.uk/fiduceo/) where an improved infrared calibration for AVHRR data was under development. SCM-5 formally ended in February 2019 when SCOPE-CM entered a new project phase (Phase III).

The planning of the next CLARA-A3 CDR will utilize the outcome of SCM-5 which so far mainly consists of an upgraded calibration method for visible channels as a result of a joint NOAA/NASA effort. The infrared calibration upgrade from the FIDUCEO project could unfortunately not be finalized and only a preliminary but incomplete (i.e., only covering about 20 years of the record) version became available in August 2018 for testing. The preliminary FIDUCEO FCDR has recently been evaluated in the CM SAF project (CM SAF, 2022). It was concluded that the FCDR quality was acceptable for the latest AVHRR/3 generation of the sensor but that large problems were found related to the harmonization of results from the previous AVHRR/2 generation.

Continued coordination of AVHRR FCDR developments beyond SCM-5 is needed to ensure the success of future reprocessing events based on the full AVHRR data record. This concerns monitoring and correction of radiances from current and future satellites still equipped with the AVHRR sensor (mainly the NOAA-19 satellite and the Metop satellites) and complementing and extending the FIDUCEO infrared calibration work for the earliest satellites (not included in FIDUCEO coverage). A first step in this direction has been taken by EUMETSAT by forming a joint EUMETSAT/CM SAF project (named the AVHRR GAC FDR project) to build upon previous experiences of the work in CM SAF, ESA Cloud_cci and FIDUCEO for improving the AVHRR GAC data record. The outcome of this project will be an official release of the AVHRR GAC FDR which also will be the basis for the next edition of CLARA (CLARA-A3). This FDR will be released in 2023, almost simultaneously with the CLARA-A3 release. A continuation of the AVHRR GAC FDR project is foreseen by EUMETSAT and the goal is to try to release a full FCDR version by 2027.

3.2.1.1.2 Adaptations to AVHRR/1.

Adding AVHRR/1 data to the AVHRR record processed by CLARA-A3 or similar CDRs means that most retrieval methods, previously used to process AVHRR/2 and AVHRR/3 data, have to be updated to cope with an AVHRR sensor with reduced capability (i.e., not including a split-window channel at 12 µm). Such adaptations have now been completed in the CM SAF project in the preparation of the CLARA-A3 CDRImportant here is to continue and finalize the stricter AVHRR FCDR work that was started within the framework of the FIDUCEO (Fidelity and uncertainty in climate data records from Earth Observations) project. This concerns monitoring and correction of radiances from current and future satellites still equipped with the AVHRR sensor (mainly the NOAA-19 satellite and the Metop satellites) and complementing and extending the FIDUCEO infrared calibration work for the earliest satellites (not included in FIDUCEO coverage). The continued work with the AVHRR FCDR is carried out by EUMETSAT and the goal is to try to release a full FCDR version by 2027. It means that it will not be available for CLARA-A3.5 but for its tentative successor (CLARA-A4).

3.2.1.1.

...

2 Adaptations to AVHRR-heritage channels on post-AVHRR sensors VIIRS and METimage

A challenge for future CDRs (beyond CLARA-A3.5 and beyond) will be the inclusion of radiances from AVHRR-heritage channels from VIIRS and METimage. This requires that appropriate measures are taken to remove effects of differences in spectral response functions between original AVHRR channels and the AVHRR-heritage channels on the new sensors. The largest differences here are found for AVHRR channel 2 with central wavelength at 0.9 µm but with a very wide spectral response (see Table 3-1) compared to corresponding channels on the new sensors. However, data from this channel is only used marginally by the CLARA-A2.1 A3 cloud masking method (affecting CFC) and not at all for the other cloud products. Consequently, adaptations and corrections here appear not to be crucial for securing the quality of the CLARA products in the future. A The first step towards the additional use of non-AVHRR sensors in the CLARA dataset here will be taken shortly to form the CLARA-A3.5 data record where AVHRR-heritage channels in VIIRS will be used to improve the coverage of observations from the afternoon orbit for the period 2012-2024. CLARA-A3.5 is planned for release in 2026. The notation 3.5 indicates that this is not a completely new CLARA edition but, at the same time, it is not just an extension since it involves some development work to define adapt methods to using the AVHRR-heritage channels from VIIRS.

Another necessary adaptation concerns the improved horizontal resolution of the original dataset from VIIRS and METimage sensors. Both sensors provide finer resolutions (better than 1 km) than the AVHRR instrument. In addition, the VIIRS instrument is using a quite advanced across-track scanning mechanism to maintain a high horizontal resolution over the full swath width. In conclusion, efforts are required to resample VIIRS and METimage data to be compatible with the original 5 km AVHRR GAC dataset. Responsible agencies (NOAA and EUMETSAT) have to be aware of the need for such a resampled radiance dataset in order to allow a proper extension of the AVHRR-based CDRs forward in time. To extend the CDR with original VIIRS and METimage data with native spectral and spatial resolution would risk creating unwanted discontinuities in the results when shifting from AVHRR to AVHRR-heritage sensors.

Initial efforts of After some initial efforts made by the ISCCP team of NOAA/NCEI has led to the definition of , a global reduced-resolution version of the VIIRS dataset with all 16 M-Bands (see Table 3-3) and with a horizontal resolution similar to GAC have been compiled. This dataset is denoted VGAC (VIIRS data resampled to AVHRR GAC resolution, Knapp et al., 2019). A commitment to produce a complete VGAC dataset (covering historic, present and future VIIRS data) for ISCCP and for other purposes has been announced by NOAA/NCEI (Andy Heidinger, NOAA, personal communication)NOAA, personal communication). However, EUMETSAT has also announced that they will also contribute in the further development and production of VGAC data, as well as developing a similar reduced resolution dataset for the METimage sensor. The VGAC dataset will be used for the previously mentioned CLARA-A3.5 data record.

...

As with the CLARA CDRs, the stability and quality of the input data is the key parameter which influences the reliability of the Cloud_cci v3 CDRs. The current AATSR TCDR is based on version 3 of the “AATSR multimission archive” maintained by CEDA and the UK National Earth Observation Data Centre (NEODC). This record incorporates the latest calibration corrections (including long-term drift corrections from vicarious calibration) and represents the most consistent and accurate record of radiances from the (A)ATSR record. A future update to this record would make a reprocessing of the Cloud_cci TCDR possible.

In  In the case of the SLSTR ICDR, the status of the level 1 radiances is considerably less stable. As of yet, there is no consistently processed version of the SLSTR radiance record, as the level 1 processing has been under development while the data has been in production. Thus, data from early in the SLSTR record has considerably worse calibration and geolocation than more recent data.

...

m channel, which, due to its location in a water-vapour absorption feature, is particularly sensitive to the presence of high-altitude clouds. Utilizing Utilising this channel in the retrieval scheme itself is unlikely to be beneficial, as accurate knowledge of the water vapour profile is needed to accurately model the radiances. However, the use of this channel in prior cloud-detection and characterization has been studied under the Cloud CCI+ project (which is a follow-on project to the original ESA-CLOUD-CCI project) and an assessment of its impact on the quality of ORAC cloud retrievals is underway.

...

Anchor
section3_3
section3_3
3.3 Methods for estimating uncertainties

3.3.1 Cloud Properties TCDR AVHRR

...

CLARA v3.0 + ICDR v3.x

In the following sub-sections, the uncertainty estimations are described for the different cloud products of the CLARA-A3 TCDR

In the following sub-sections, the uncertainty estimations are described product by product. However, generally CLARA-A3 uncertainty estimates will be expanded by considering uncertainty in ancillary data used in the retrieval process, incl. surface albedo, emissivity and temperature and atmospheric (trace) gas amounts. Furthermore, the aggregation to Level 3 will be refined according to methods described in Stengel et al. (2017).

3.3.1.1 Cloud Fractional Cover (CFC)

The current CLARA-A2.1 CFC products are not associated with any uncertainty estimates. Thus, uncertainty information is only available as results achieved by associated validation activities. Some quality flagging is applied in the original PPS cloud masking product but this information is not propagated or compiled for the Level 3 products.

For CLARA-A3, Bayesian cloud masking will be used which enables estimation of the uncertainty of cloud masking based on the used spectral information. However, this uncertainty will not consider uncertainty in input data (AVHRR radiances or auxiliary data). Further work is needed here (e.g., Monte Carlo simulations) for potential CLARA editions beyond CLARA-A3.

3.3.1.

...

1 Cloud

...

Fractional Cover (

...

CFC)

CFC uncertainties can be interpreted directly from the CMAPROB product which is provided together with the binary cloud mask. Maximum uncertainty is found at the 50% cloud probability level. For the level-3 product, a simple estimation based on the averaging of the probability distance from the 50% threshold for clear and cloudy pixels is provided.

3.3.1.2 Cloud Top Pressure (CTP)

For estimating the CTP uncertainty, the absolute CTP difference from the 16th and the 84th percentile is provided. These CTP 1-sigma uncertainties from Level-2b files are then propagated into level-3 products, following Stengel et al. (2017)

The current CLARA-A2.1 CTP product is not associated with any uncertainty estimates. Thus, uncertainty information is only provided as results achieved by associated validation activities. One aspect leading to additional uncertainty and an inconsistency with the CFC product is the fact that not all pixels declared cloudy in the basic PPS cloud mask product will be given a valid CTP result. This concerns very thin high clouds and some fractional (sub-pixel) clouds at all altitudes. Pixels with lacking CTP results amount to about 2-3 % of all pixels declared cloudy.

For CLARA-A3 limited uncertainty information (i.e., percentile distributions) will be provided in original Level 2 products and this can be utilized to estimate the uncertainty of Level 3 products. Also, all cloudy pixels will be associated with a valid CTP result in CLARA-A3.

3.3.1.3 Liquid Water Path (LWP)

...

Estimated uncertainties in reflectance measurements and various input variables (e.g., surface albedo, total ozone column) are propagated to yield uncertainty estimates in retrieved COT and CER parameters. These are, in turn, propagated to uncertainty estimates in LWP and IWP. Further details are given in NWC SAF (2021). Note, however, that uncertainties do not include deviations from the assumption of horizontal and vertical homogeneity of the clouds as a source of error. Error estimates are propagated into level-3 products, following Stengel et al. (2017). Uncertainties are aggregated to Level 3 by linear averaging.

3.3.1.4 Ice Water Path (IWP)

Uncertainty estimates for IWP in CLARA-A2.1 and CLARA-A3 are /will be derived in the same way as for LWP.

...

C3S products from Cloud_cci data and the SLSTR ICDR follow the methodology described by Stengel et al. (2018), with four separate uncertainty parameters being included for each mean parameter, x:

Uncertainty parameterFormulaDescription
  1. The standard deviation of the retrieval parameter,

    Mathinline
    \sigma_{std}

    , which is the square-root of the variance, defined by:


Mathinline
\sigma_{std}^{2} = \frac{1}{N}\sum\limits_{i=1}^{N}(x_{i} - \langle x \rangle)^{2} \ \ (Eq 2)


where N is the number of L2 pixels included in the L3 average, 

Mathinline
\sigma_i

is the uncertainty on an individual L2 value and c is the correlation between the L2 pixels. The reader is referred to Stengel et al. (2018)

2. The mean uncertainty of the parameter,

Mathinline
\langle \sigma \rangle

, which is simply the mean of the uncertainty values of the data included in the L3 mean:


Mathinline
\langle \sigma \rangle = \frac{1}{N}\sum\limits_{i=1}^{N}(\sigma) \ \ (Eq 3)


3. The propagated uncertainty (assuming independent measurements),

Mathinline
\sigma_{prop}

, which is calculated from the mean of the squared uncertainties of the uncertainty values of the data included in the L3 mean:


Mathinline
\sigma_{prop}^{2} = \frac{1}{N} \langle \sigma_{i}^{2} \rangle \ \ (Eq 4)


4. Correlated uncertainty (including correlation between the L2 pixels within the L3 average),

Mathinline
\sigma_{corr}

, which is given by the expression:


Mathinline
\sigma_{corr}^{2} = \sigma_{std}^{2} - (1-c) \sqrt{\langle \sigma_{i}^{2} \rangle - \langle \sigma_{i} \rangle^{2}} \ \ (Eq 5)


Anchor
section3_4
section3_4
3.4 Opportunities to improve quality and fitness-for-purpose of the CDRs

3.4.1 Cloud Properties TCDR AVHRR CLARA

...

v3.0 + ICDR

...

v3.x

Sections 3.2 and 3.3 have already indicated some of the improvements of CLARA cloud CDRs which will be included in the upcoming introduced in CLARA-A3 CDR. The most prominent updates here are the introduction of Bayesian (probabilistic) cloud masking (Karlsson et al., 2015, 2020) and CTP retrievals based on artificial neural networks (Håkansson et al., 2018). These changes are largely made possible by the access to high-quality cloud information from the CALIPSO-CALIOP sensor since 2006 (Winker et al., 2009).

The high potential of using artificial neural network methods for improving the cloud product quality is not only demonstrated for the planned CLARA-A3 CTP product but also for products like cloud masking and cloud phase retrievals (Stengel et al., 2017, 2020 and Sus et al., 2018). Also, this development is a consequence of the availability of the CALIPSO-CALIOP observations. Continuing development efforts here (e.g. within the ESA CCI+ project) efforts here will most likely impact future compilation of AVHRR-based CDRs. A central question here will be to find out if ANN-based methods are able to match physical retrieval methods (e.g., as described by McGarragh et al., 2018).

An interesting extension of the an AVHRR FCDR was proposed implemented by NOAA in the SCM-5 project. NOAA proposed to add selected information from the PATMOS-x v6 CDR, released in 2023 (Foster et al., 2023). Here, information was added to the AVHRR FCDR from High-resolution Infrared Radiation Sounder (HIRS) instrument to the AVHRR radiance dataset in order to improve in particular the cloud top pressure retrieval. HIRS is another instrument which has been flying on the same satellite platforms as the AVHRR instrument historically but for the purpose of providing coarse resolution vertical sounding information. Current AVHRR-based cloud top retrieval methods have been shown to largely underestimate cloud top heights of thin cirrus clouds. At the same time, it is well-known that the HIRS measurements are very sensitive to the existence of thin Cirrus clouds. Thus, to merge selected HIRS information (e.g., channels near 7 µm and 13 µm) with AVHRR data would potentially improve both thin Cirrus detection and the retrieval of cloud top heights for thin Cirrus clouds. A fusion method applicable for sensors like HIRS and AVHRR has been proposed by Weisz et al. (2017) and the method will be applied was used in the next PATMOS-X v6 dataset (as mentioned by Heidinger et al. , 2014). However, the proposed fusion technique is not the only possible approach here so further research is needed to identify the most efficient method to merge information from the two sensors. In addition, just as for AVHRR-heritage applications, methods must be developed to cope with the transfer from the HIRS sounding instrument to corresponding HIRS-heritage instruments on the JPSS and EPS-SG satellites to enable a consistent use over the full time period of possible AVHRR and AVHRR-heritage measurements.

...

Anchor
section3_5
section3_5
3.5 Scientific Research needs

3.5.1 Cloud Properties TCDR AVHRR CLARA

...

v3.0 + ICDR

...

v3.x

Regarding the development of processing algorithms, further improvement in this field will be linked to the availability of high-quality reference measurements from active sensors, like the CALIPSO-CALIOP sensor. As the The access to such reference data is not guaranteed (e.g., by further delay or problems with reference data from CALIPSO ceased in summer 2022 and similar data from the upcoming EarthCARE mission ), development may be slowed down. This is especially is not yet available (EarthCARE launch currently expect in summer 2024). This data gap could be serious for sensors like METimage which most likely will not have any has no overlap with CALIPSO-CALIOP data (in contrast to VIIRS on Suomi-NPP and NOAA-20, where we have more than 10 years of useful CALIPSO data). However, since also the launch of the satellites in EPS-SG with METimage onboard is delayed (expected in 2025) this problem is less serious (provided that the EarthCARE launch is successful and on schedule).

New algorithm developments for retrieval of cloud parameters are likely to be based on machine learning approaches where also uncertainty measures will be added If the reference data availability is guaranteed, the next question is whether Bayesian cloud masking methods and/or physical retrieval methods including proper error propagation modelling (e.g., McGarragh et al., 2018) will be able to cope with the results provided by ANN-based methods. The latter have shown remarkable progress in generating various high-quality cloud products but they still lack appropriate methods for estimating the uncertainty of the products. However, promising attempts (e.g., as demonstrated by Pfreundschuh et al., 2018) to address this problem are under consideration. Intensive research in this field is expected to continueal., 2018).

Finally, a limiting factor for all climate monitoring activities from space is the very high requirement on radiometric calibration in order to be able to distinguish a climate trend from trends caused by instrumental inhomogeneities, errors or noise. Further work on achieving high quality radiance references is therefore needed in addition to securing the availability of active cloud sensing instruments in space.

3.5.2 Cloud Properties TCDR ESA AATSR v3.0 + ICDR ESA SLSTR v3.x

...

3.6 Opportunities from exploiting the Sentinels and any other relevant satellite

3.6.1 Cloud Properties TCDR AVHRR CLARA

...

v3.0 + ICDR

...

v3.x

The Sentinel-3 mission carries the SLSTR and OLCI instruments which both have several AVHRR-heritage channels. In principle, data from these two instruments could be used to generate the same kind of cloud products as for AVHRR. For example, a new version the latest versions of the NWC SAF PPS cloud processing package (to be used for compiling the CLARA-A3 TCDR) is already capable of processing SLSTR data. This information could thus be added to the other products generated from pure AVHRR-data and from the previously mentioned AVHRR-heritage sensors.

However, since Sentinel-3 is orbiting in a morning orbit (daytime equator crossing time at 10 am) the observations unfortunately tend to duplicate observations from other satellites (e.g., Metop satellites and EPS-SG satellites). Thus, the added value could be questioned (unless some of the other morning satellites fail). This question could be broadened to also include the information from other polar orbiting satellites, like Aqua and Terra with MODIS data and the Chinese polar orbiting FY-3D+ satellites with data from the MERSI-2 imager (the latter now also handled by the latest NWC SAF PPS package). Again, it is clear that the satellite orbits used for these satellites are practically identical to the morning and afternoon satellites of the NOAA and Metop series of satellites and their successors. Thus, the added value from these observations will probably only be marginal.

More important seems to be to build the dataset consistently on observations from one morning satellite and one afternoon satellite (i.e., the prime satellites as demonstrated by by Stengel et al., 2017). A CDR based on only these prime satellites appears to be the most useful dataset from a climate monitoring perspective. Other satellites outside the primary AVHRR and AVHRR-heritage family could mainly serve as an important backup source of information if some of the prime satellites fail.

3.6.2 Cloud Properties TCDR ESA AATSR v3.0 + ICDR ESA SLSTR v3.1.1+v4.0

The ESA SLSTR v3.x ICDR directly exploits data from the Sentinel-3 platform. There have been examples shown of utilizing Sentinel-3 OLCI-like measurements (mainly using MERIS on ENVISAT) for cloud retrieval in conjunction with (A)ATSR or SLSTR (Carbajal Henken et al. 2014), but difficulties in cross-calibration and co-registration of the different instruments have meant these products have not shown improved performance over the (A)ATSR/SLSTR only algorithms. The availability of a well co-located and calibrated joint SLSTR-OLCI L1 product, could resurrect this approach to further improving cloud products derived from Sentinel-3 (and the preceding ENVISAT).

As discussed in in section 3.4.2, the ORAC retrieval scheme can be, and has been, applied to a wide range of satellite visible-IR imaging radiometers. A particular instrument, of direct relevance to the Sentinel satellite program, is the Flexible Combined Imager (FCI) to fly on MeteoSat Third Generation/Sentinel-4. This instrument is essentially a replacement for the SEVIRI sensors on MSG, with capabilities similar to those provided by Himawari-AHI and GOES-ABI imagers (which ORAC has already been applied to).

...

Baum B.A. et al (2010), The impact of ice particle roughness on the scattering phase matrix, Journal of Quantitative Spectroscopy and Radiative Transfer, Volume 111, Issues 17–18, DOI: https://doi.org/10.1016/j.jqsrt.2010.07.008.

Baum B.A. et al (2014), Ice cloud single-scattering property models with the full phase matrix at wavelengths from 0.2 to 100µm, Journal of Quantitative Spectroscopy and Radiative Transfer, Volume 146,DOI: https://doi.org/10.1016/j.jqsrt.2014.02.029.

Brodzik, M. J. and J. S. Stewart. 2016: Near-Real-Time SSM/I-SSMIS EASE-Grid Daily Global Ice Concentration and Snow Extent, Version 5. Boulder, Colorado USA. NASA National Snow and Ice Data Center Distributed Active Archive Center. doi: https://doi.org/10.5067/3KB2JPLFPK3R.

Carbajal Henken, C.K., R. Lindstrot, R. Preusker and J. Fischer, 2014: FAME-C: cloud property retrieval using synergistic AATSR and MERIS observations. Atmos. Meas. Tech., 7, 3873–3890, doi:DOI: https://doi.org/10.5194/amt-7-3873-2014.

CM SAF, 2022: Evaluation of the preliminary release of the FIDUCEO AVHRR GAC FCDR, SAF/CM/SMHI/REP/FDR, v. 1.0, (submitted to EUMETSAT).

Dee et al. (35 named authors), 2011: The ERA-Interim reanalysis: configuration and performance of the data assimilation system, Quart. J. Royal Met. Soc., 137 (656), 553 - 597. Online version: http://onlinelibrary.wiley.com/doi/10.1002/qj.828/abstract.

Devasthale, A.; Karlsson, K.-G. Decadal Stability and Trends in the Global Cloud Amount and Cloud Top Temperature in the Satellite-Based Climate Data Records. Remote Sens. 2023, 15, 3819. DOI: https://doi.org/10.3390/rs15153819

EUMETSAT, 2023a: AVHRR Fundamental Data Record – Release 1 – Multimission, European Organisation for the Exploitation of Meteorological Satellites [data set], DOI: https://doi.org/10.15770/EUM_SEC_CLM_0060, 2023a

EUMETSAT, 2023bEUMETSAT, 2023 : PyGAC AVHRR FDR Release 1 – Product Users Guide, Document EUM/OPS/DOC/21/1269950, DOI : DOI: https://doi.org/10.15770/EUM_SEC_CLM_0060.

Stoffelen, A., 1998: Towards the true near-surface wind speed: Error modeling and calibration using triple collocation. J. Geophys. Res. 103C3, 7755-7766. doi: https://doi.org/10.1029/97JC03180

Dybbroe, A., A. Thoss and K.-G. Karlsson, 2005: NWCSAF AVHRR cloud detection and analysis using dynamic thresholds and radiative transfer modeling - Part I: Algorithm description, J. Appl. Meteor, 44, 39-54. doi: https://doi.org/10.1175/JAM-2188.1

Eidenshink, J. and Faundeen, J., 1994: The 1 km AVHRR global land data set-first stages in implementation. Int. J. Rem. Sens., 15(17), 3443-3462. EUMETSAT OSI SAF, 2015: Global sea ice concentration climate data records 1978-2015 (v1.2, 2015), [Online]. Norwegian and Danish Meteorological Institutes. doi: 10.15770/EUM_SAF_OSI_0001 and 10.15770/EUM_SAF_OSI_0005doi: https://doi.org/10.1080/01431169408954339

Foster, M. J., C. Phillips, A. K. Heidinger, E. E. Borbas, Y. Li, W. P. Menzel, A. Walther, and E. Weisz, 2023: PATMOS-x Version 6.0: 40 Years of Merged AVHRR and HIRS Global Cloud Data. J. Climate, 36, 1143–1160, DOI: https://doi.org/10.1175/JCLI-D-22-0147.1.

Han, Q., W.B. Rossow and A.A. Lasis, 1994: Near-Global Survey of Effective Droplet Radii in Liquid Water Clouds using ISCCP Data, J. Climate, 7, 465-497, 1994. doi: https://doi.org/10.1175/1520-0442(1994)007%3C0465:NGSOED%3E2.0.CO;2

Heidinger, A. K.: Climate Algorithm Theoretical Basis Document (C-ATBD): Fundamental Climate Data Record (CDR) of Reflectance from AVHRR Bands 1, 2 and 3a, CDR Program Document Number CDRP-ATBD-0184, Revision 2, 2018Heidinger, A.K.; Foster, M.J.; Walther, A.; Zhao, Z., 2014: The Pathfinder Atmospheres Extended (PATMOS-x) AVHRR climate data set. Bull. Am. Meteorol. Soc., 95, 909–922.

Heidinger, A.K., W.C. Straka, C.C. Molling, J.T. Sullivan and X.Q. Wu, 2010: Deriving an inter-sensor consistent calibration for the AVHRR solar reflectance data record. Int. J. Rem. Sens., 31(24), 6493-6517, doi: https://doi.org/10.1080/01431161.2010.496472

Hersbach, H., Bell, B., Berrisford, P., Hirahara, S., Horányi, A., Muñoz-Sabater, J., ... & Thépaut, J. N. 2020: The ERA5 global reanalysis. Quarterly Journal of the Royal Meteorological Society, 146(730), 1999-2049, doi: https://doi.org/10.1002/qj.3803

Håkansson, N., Adok, C., Thoss, A., Scheirer, R., and Hörnquist, S., 2018: Neural network cloud top pressure and height for MODIS, Atmos. Meas. Tech., 11, 3177-3196, DOI: https://doi.org/10.5194/amt-11-3177-2018.

Karlsson, K.-G. , Anttila, K., Trentmann, J., Stengel, M., Fokke Meirink, J., and Devasthale, A., Hanschmann, T., Kothe, S., Jääskeläinen, E., Sedlar, J., Benas, N., van Zadelhoff, G.-J., Schlundt, C., Stein, D., Finkensieper, S., Håkansson, N., and Hollmann, R.: CLARA-A2: the second edition of the CM SAF cloud and radiation data record from 34 years of global AVHRR data, Atmos. Chem. Phys., 17, 5809-58282018: Inter-Comparison and Evaluation of the Four Longest Satellite-Derived Cloud Climate Data Records: CLARA-A2, ESA Cloud CCI V3, ISCCP-HGM, and PATMOS-x, Rem. Sens., 10, 1567, DOI: https://doi.org/10.3390/rs10101567

Karlsson, K.-G. and Håkansson, N.: Characterization of AVHRR global cloud detection sensitivity based on CALIPSO-CALIOP cloud optical thickness information: Demonstration of results based on the CM SAF CLARA-A2 climate data record. Atmos. Meas. Tech. 2018, 11, 633–649, https://doi.org/10.5194/acpamt-1711-5809633-20172018, 20172018.

Karlsson, K.-G. and Devasthale, A., 2018: Inter-Comparison and Evaluation of the Four Longest Satellite-Derived Cloud Climate Data Records: CLARA-A2, ESA Cloud CCI V3, ISCCP-HGM, and PATMOS-x, Rem. Sens., 10, 1567, doi:10.3390/rs10101567, Johansson, E., Håkansson, N., Sedlar, J. and Eliasson, S.: Probabilistic Cloud Masking for the Generation of CM SAF Cloud Climate Data Records from AVHRR and SEVIRI Sensors. Remote Sens.12, 713. https://doi.org/10.3390/rs12040713, 2020.

Karlsson, K.-G., Håkansson, N., Mittaz, J. P. D., Hanschmann, T. and Devasthale, A., 2017: Impact of AVHRR Channel 3b Noise on Climate Data Records: Filtering Method Applied to the CM SAF CLARA-A2 Data Record, Rem. Sens., 9, 568, doi: 10.3390/rs9060568.Karlsson, K.-G., E. Johansson and A. Devasthale, 2015: Advancing the uncertainty characterisation of cloud masking in passive satellite imagery: Probabilistic formulations for NOAA AVHRR data, Rem. Sens. Env. , 158, 126-139; doi:10.1016/j.rse.2014.10.028DOI: https://doi.org/10.3390/rs9060568.

Karlsson, K.-G., Johansson, E., Håkansson, N., Sedlar, J. and Eliasson, S., 2020: Probabilistic cloud masking for the generation of CM SAF cloud climate data records from AVHRR and SEVIRI sensors, accepted for publication in Rem. Sens.

McGarragh, G.R.; Poulsen, C.A.; Thomas, G.E.; Povey, A.C.; Sus, O.; Stapelberg, S.; Schlundt, C.; Proud, S.; Christensen, M.W.; Stengel, M.; et al., 2018: The Community Cloud retrieval for CLimate (CC4CL). Part II: The optimal estimation approach. Atmos. Meas. Tech., 11, 3397–3431.

, doi: https://doi.org/10.3390/rs12040713

Karlsson, K.-G., Stengel, M., Meirink, J. F., Riihelä, A., Trentmann, J., Akkermans, T., Stein, D., Devasthale, A., Eliasson, S., Johansson, E., Håkansson, N., Solodovnik, I., Benas, N., Clerbaux, N., Selbach, N., Schröder, M., and Hollmann, R.: CLARA-A3: The third edition of the AVHRR-based CM SAF climate data record on clouds, radiation and surface albedo covering the period 1979 to 2023, Earth Syst. Sci. Data, 15, 4901–4926, DOI: https://doi.org/10.5194/essd-15-4901-2023, 2023.

Knapp, K., Young, A., Inamdar, A. and Hankins, W., 2019: ISCCP and VIIRS: Adapting new instruments to extend historical climate data records—VGAC, https://ams.confex.com/ams/JOINTSATMET/mediafile/Handout/Paper360556/VGAC%20version%201%20-%20poster.pdfMoody, E.G., M.D. King, C.B. Schaaf, and S. Platnick, 2008: MODIS-Derived Spatially Complete Surface Albedo Products: Spatial and Temporal Pixel Distribution and Zonal Averages. J. Appl. Meteorol. Clim., 47, 2879–2894.

Nakajima, T., and M. D. King, 1990: Determination of the Optical Thickness and Effective Particle Radius of Clouds from Reflected Solar Radiation Measurements. Part 1: Theory. J. Atmos. Sci ., 47, 1878-1893Measurements. Part 1: Theory. J. Atmos. Sci., 47, 1878-1893, doi: https://doi.org/10.1175/1520-0469(1990)047%3C1878:DOTOTA%3E2.0.CO;2

NWC SAF: Algorithm Theoretical Basis Document for Cloud Micro Physics of the NWC/PPS, EUMETSAT Satellite Application Facility on Nowcasting and Very Short Range Forecasting, NWC/CDOP3/PPS/SMHI/SCI/ATBD/CMIC, Issue 3.0, 12 October 2021.

Pavalonis, M.J., A.K. Heidinger and T. Uttal, 2005: Daytime Global Cloud Typing from AVHRR and VIIRS: Algorithm Description, Validation and Comparisons. J. Appl. Meteor., 44, 804-826. DOI: https://doi.org/10.1175/JAM2236.1.

Pfreundschuh, S., Eriksson, P., Duncan, D., Rydberg, B., Håkansson, N., and Thoss, A., 2018: A neural network approach to estimating a posteriori distributions of Bayesian retrieval problems, Atmos. Meas. Tech., 11, 4627-4643, DOI: https://doi.org/10.5194/amt-11-4627-2018.

Poulsen, C.A, G. McGarragh, M. Stengal, 2018: ESA Cloud_cci Validation Report for MODIS multi-layer cloud. Report, accessed 19-Dec-2019: http://www.esa-cloud-cci.org/sites/default/files/upload/Cloud_cci_D4.1.2_RMLEV_v1.1.pdf

Schaaf, C. and Wang, Z., 2015: MCD43C1 MODIS/Terra+Aqua BRDF/AlbedoModel Parameters Daily L3 Global 0.05Deg CMG V006, NASA EOSDIS Land Processes DAAC, DOI: https://doi.org/10.5067/MODIS/MCD43C1.006.

Smith D. 2012: ENVISAT AATSR Instrument Performance – End of Mission Report. Report, accessed 02-Mar-2020: https://atsrsensors.org/pdf/ENVISAT%20AATSR%20Instrument%20Performance%20-%20End%20of%20Mission%20Report%20%20-%20Issue%201.0.pdf

Seemann, S.W., E. E. Borbas, R. O. Knuteson, G. R. Stephenson, H.-L. Huang, 2008: Development of a Global Infrared Land Surface Emissivity Database for Application to Clear Sky Sounding Retrievals from Multi-spectral Satellite Radiance Measurements, J. Appl. Meteorol. Clim., 47, 108-123, doi: https://doi.org/10.1175/2007JAMC1590.1

Stengel, M., Stapelberg, S., Sus, O., Finkensieper, S., Würzler, B., Philipp, D., Hollmann, R., Poulsen, C., Christensen, M.; and McGarragh, G., 2020: Cloud_cci Advanced Very High Resolution Radiometer post meridiem (AVHRR-PM) dataset version 3: 35 year climatology of global cloud and radiation properties, Earth Syst. Sci. Data, 12, 41-60, DOI: https://doi.org/10.5194/essd-12-41-2020.

Stengel, M, C. Poulsen, R. Hollmann, N. Schneider, L. Gómez Chova, O. Sus, C. Schlundt, S. Testorp, 2018: ESA Cloud_cci Comprehensive Error Characterisation Report, Issue 4, Rev. 1. Accessed 19-Dec-2019: DOI: http://www.esa-cloud-cci.org/sites/default/files/upload/Cloud_cci_D1.3_CECR_v4.1.pdf

Stengel, M.; Stapelberg, S.; Sus, O.; Schlundt, C.; Poulsen, C.; Thomas, G.; Christensen, M.; Carbajal Henken, C.; Preusker, R.; Fischer, J.; et al., 2017: Cloud property datasets retrieved from AVHRR, MODIS, AATSR and MERIS in the framework of the Cloud_cci project. Earth Syst. Sci. Data, 9, 881–904.. Sci. Data, 9, 881–904, doi: https://doi.org/10.5194/essd-9-881-2017

Stephens, G.., 1978: Radiation profiles in extended water clouds. II: Parameterization schemes. J. Atmos. Sci., 35, 2123-2132, doi: https://doi.org/10.1175/1520-0469(1978)035%3C2123:RPIEWC%3E2.0.CO;2

Stoffelen , A., 1998: Towards the true near-surface wind speed: Error modeling and calibration using triple collocation. J. Geophys. Res. 103C3, 7755-7766, doi: https://doi.org/10.1029/97JC03180

Sun, B., Free, M., Yoo, H. L., Foster, M. J., Heidinger, A. and Karlsson, K.-G., 2015: Variability and Trends in U.S. Cloud Cover: ISCCP, PATMOS-x and CLARA-A1 Compared to HomegeneityHomogeneity-Adjusted Weather Observations, J. Clim., 28, 4373-4389, DOI: https://doi.org/10.1175/JCLI-D-14-00805.1D-14-00805.1.

Sun, Q., Wang, Z., Li, Z., Erb, A., and Schaaf, C. L. B., 2017: Evaluation of the global MODIS 30 arc-second spatially and temporally complete snow-free land surface albedo and reflectance anisotropy dataset. Int. J. Appl. Earth Obs., 58, 36-49, doi: https://doi.org/10.1016/j.jag.2017.01.011.

Sus, O.; Stengel, M.; Stapelberg, S.; McGarragh, G.; Poulsen, C.; Povey, A.C.; Schlundt, C.; Thomas, G.; Christensen, M.; Proud, S.; et al., 2018: The Community Cloud retrieval for Climate (CC4CL). Part I: A framework applied to multiple satellite imaging sensors. Atmos. Meas. Tech., 11, 3373–3396, doi: https://doi.org/10.5194/amt-11-3373-2018

USGS, 1996: Global 30 arc seconds topography database, GTOPO30, (http://edcdaac.usgs.gov/gtopo30/)

Wan, Z., Hook, S., Hulley, G., 2015: MYD11C3 MODIS/Aqua Land Surface Temperature/Emissivity Monthly L3 Global 0.05Deg CMG V006 Winker, D.: CALIPSO LID L2 5km Standard HDF File - Version 4.10 [Data set]. NASA EOSDIS LP Langley Research Center Atmospheric Science Data Center DAAC. doi: DOI: https://doi.org/10.5067/MODIS/MYD11C3.006CALIOP/CALIPSO/LID_L2_05kmCLay-Standard-V4-20, 2016.

Winker, D. M., Vaughan, M. A., Omar, A., Hu, Y. and Powell, K. A., 2009: Overview of the CALIPSO mission and CALIOP data processing algorithms, J. Atmos. Oceanic. Technol., 26, 2310-2323, doi: https://10.1175/2009JTECHA1281.1.



Info

This document has been produced with funding by the European Union in the context of the Copernicus Climate Change Service (C3S).The activities leading to these results have been contracted , operated by the European Centre for Medium-Range Weather Forecasts , operator of C3S on behalf of on the European Union (Delegation Agreement signed on 11/11/2014 and Contribution Agreement signed on 22/07/2021). All information in this document is provided "as is" “as is” and no guarantee or of warranty is given that the information is fit for any particular purpose. The users thereof use the information at their sole risk and liability. For the avoidance of all doubt, the European Commission and the European Centre for Medium-Range Weather Forecasts have no liability in respect of this document, which is merely representing the

author's

author’s view.


Related articles

Content by Label
max3
cqllabel in ("ecv","cloud_properties")