Contributors: M. Reuter (Institute of Environmental Physics (IUP)), B. Fuentes Andrade (Institute of Environmental Physics (IUP)), M. Buchwitz (Institute of Environmental Physics (IUP))
Issued by: Institute of Environmental Physics (IUP), University of Bremen / Maximilian Reuter
Date: 19/11/2024
Ref: C3S2_313a_DLR_WP1-DDP-GHG-v1_PQAR_XGHG_v4.6
Official reference number service contract: 2024/C3S2_313a_DLR/SC1
History of modifications
List of datasets covered by this document
Acronyms
General definitions
Essential climate variable (ECV): An ECV is a physical, chemical, or biological variable or a group of linked variables that critically contributes to the characterization of Earth's climate (Bojinski et al., 2014).
Climate data record (CDR): The US National Research Council (NRC) defines a CDR as a time series of measurements of sufficient length, consistency, and continuity to determine climate variability and change (National Research Council, 2004).
Fundamental climate data record (FCDR): A fundamental climate data record (FCDR) is a CDR of calibrated and quality-controlled data designed to allow the generation of homogeneous products that are accurate and stable enough for climate monitoring.
Thematic climate data record (TCDR): A thematic climate data record (TCDR) is a long time series of an essential climate variable (ECV) (Werscheck, 2015).
Intermediate climate data record (ICDR): An intermediate climate data record (ICDR) is a TCDR which undergoes regular and consistent updates (Werscheck, 2015), for example because it is being generated by a satellite sensor in operation.
Satellite data processing levels: The NASA Earth Observing System (EOS) distinguishes six processing levels of satellite data, ranging from Level 0 (L0) to Level 4 (L4) as follows (Parkinson et al., 2006).
L0 |
Unprocessed instrument data |
L1A |
Unprocessed instrument data alongside ancillary information |
L1B |
Data processed to sensor units (geo-located calibrated spectral radiance and solar irradiance) |
L2 |
Derived geophysical variables (e.g., XCO2) over one orbit |
L3 |
Geophysical variables averaged in time and mapped on a global longitude/latitude horizontal grid |
L4 |
Model output derived by assimilation of observations, or variables derived from multiple measurements (or both) |
Absolute systematic error or systematic error: Component of measurement error that in replicate measurements remains constant or varies in a predictable manner. Note that "systematic error" refers to the absolute systematic error (in contrast to "relative systematic error" defined below). For satellite GHG ECV products especially the relative systematic error is important.
Relative systematic error, relative accuracy or relative bias: Identical with "Systematic error" but after bias correction and without considering a possible global offset (overall mean bias). Reflects the importance of spatially and temporally correlated errors (spatio-temporal biases). Computed from standard deviations of spatial and temporal biases.
Bias: Estimate of a systematic measurement error.
Precision: Measure of reproducibility or repeatability of the measurement without reference to an international standard so that precision is a measure of the random and not the systematic error. Suitable averaging of the random error can improve the precision of the measurement but does not establish the systematic error of the observation (CMUG-RBD, 2012).
Note: Precision is quantified with the standard deviation (1-sigma) of the error distribution.
Stability: Term often invoked with respect to long-term records when no absolute standard is available to quantitatively establish the systematic error - the bias defining the time-dependent (or instrument-dependent) difference between the observed quantity and the true value (CMUG-RBD, 2012).
Note: Stability requirements cover inter-annual error changes. If the change in the average bias from one year to another is larger than the defined values, the corresponding product does not meet the stability requirement.
Representativity: Extent to which an average of a set of measured values corresponds to the true average, e.g., over a grid cell. It is important when comparing with or assimilating in models. Measurements are typically averaged over different horizontal and vertical scales compared to model fields. If the measurements are smaller scale than the model it is important. The sampling strategy can also affect this term (CMUG-RBD, 2012).
Threshold requirement: The threshold is the limit at which the observation becomes ineffectual and is not of use for climate-related applications (CMUG-RBD, 2012).
Goal requirement: The goal is an ideal requirement above which further improvements are not necessary (CMUG-RBD, 2012).
Breakthrough requirement: The breakthrough is an intermediate level between the "threshold" and "goal" requirements, which - if achieved - would result in a significant improvement for the targeted application. The breakthrough level may be considered as an optimum, from a cost-benefit point of view when planning or designing observing systems (CMUG-RBD, 2012).
Horizontal resolution: Area over which one value of the variable is representative of (CMUG-RBD, 2012).
Vertical resolution: Height over which one value of the variable is representative of. Only used for profile data (CMUG-RBD, 2012).
Observing Cycle (or Revisit Time): Temporal frequency at which the measurements are required (CMUG-RBD, 2012).
XCO2 (column-averaged dry-air mole fraction of atmospheric carbon dioxide): amount of CO2, expressed in moles, in the vertical column divided by the amount of dry air, also expressed in moles, in that vertical column.
XCH4 (column-averaged dry-air mole fraction of atmospheric methane): amount of CH4, expressed in moles, in the vertical column divided by the amount of dry air, also expressed in moles, in that vertical column.
Column averaging kernel: The column averaging kernel vertical profile represents the sensitivity of the retrieved XCO2 or XCH4 to the true mole fraction depending on altitude. Values near one are ideal and indicate that the influence of the a priori is minimal.
A priori profile: The CO2 or CH4 a priori profile represents the knowledge of the vertical profile of the dry-air mole fraction of CO2 or CH4 before the measurement. See Rodgers, 2000 for a more detailed explanation.
Observations for Model Intercomparison Project (Obs4MIPs, here also OBS4MIPS): is an effort to make observational data more accessible for climate model evaluation, development and research. An Obs4MIPs (or OBS4MIPS) dataset is a dataset technically aligned with climate model data, and following data specifications consistent with the CMIP (Coupled Model Intercomparison Project) standard output. See https://pcmdi.github.io/obs4MIPs/ for more detailed information.
Glint observation mode: Viewing geometry used by some satellite instruments where the detector points toward the direction of the specularly reflected sunlight, i.e. the viewing zenith angle and solar zenith angle are approximately equal. This mode is used for measuring CO2 and CH4 over water surfaces (such as the ocean), which have low reflectivity in the spectral region used for the retrieval of these gases. By observing in glint mode, the instrument measures a higher reflected radiance compared to other viewing geometries, enhancing the signal for the gas retrievals.
Inter-algorithm spread (IAS): Algorithm-to-algorithm standard deviation of the grid box averages (for a set of L3 algorithms). It informs about potential regional or temporal systematic uncertainties.
Mean Local Time (MLT): Expression of time given by the hour angle of the mean position of the Sun, plus an offset of 12 hours.
Executive summary
This document is a Product Quality Assessment Report (PQAR) generated in the framework of the Copernicus Climate Change Service (C3S, https://climate.copernicus.eu/). For C3S a large number of satellite-derived Essential Climate Variable (ECV) data products are generated and made available via the Copernicus Climate Data Store (CDS, https://cds.climate.copernicus.eu/).
This document describes how the quality of two satellite-derived atmospheric carbon dioxide (CO2) and methane (CH4) data products has been assessed. The two products are column-averaged dry-air mole fractions of CO2 and CH4, denoted as XCO2 (in parts per million, ppm) and XCH4 (in parts per billion, ppb), respectively. These two "XGHG" products are generated by the Institute of Environmental Physics (IUP), University of Bremen on behalf of the C3S. In the following this project is referred to as C3S/GHG project. Within this project also mid-tropospheric CO2 and CH4 products are generated. These mid-tropospheric CO2 and CH4 products are not described in this document but in dedicated separate documents (ATBD MTGHG, 2024, PQAR MTGHG, 2024 and PUGS MTGHG, 2024).
The two data products assessed in this document (XCO2_OBS4MIPS and XCH4_OBS4MIPS), are merged multi-sensor XCO2 and XCH4 Level 3 (L3) products with monthly time and 5ox5o spatial resolution, which are generated using an ensemble of individual satellite sensor Level 2 (L2) products. The algorithm that produced these products has been developed at the University of Bremen, Germany, and has also been used in the past to generate previous versions of these products. The main changes of the current products with respect to past versions are the updated input data and extended time period. See PUGS XGHG, 2024 for more details on the history of changes in previously generated datasets.
Specifically, this Product Quality Assessment Report (PQAR) at hand describes how version 4.6 of XGHG products XCO2_OBS4MIPS and XCH4_OBS4MIPS has been validated using XCO2 and XCH4 retrievals from the Total Carbon Column Observing Network (TCCON, https://tccon.caltech.edu/).
Summary product quality XCO2_OBS4MIPS v4.6:
The TCCON-validation of the XCO2_OBS4MIPS v4.6 product can be summarized as follows. There is only a small overall average bias of 0.34 ppm. Regional biases estimated from the site-to-site bias variability amount to 0.30 ppm. The average seasonal and spatiotemporal bias amounts to 0.26 ppm and 0.40 ppm, respectively. The overall linear drift of 0.02 ppm/year is much smaller than its site-to-site variability of 0.12 ppm/year and, therefore it is not significant. The uncertainty ratio, i.e. the ratio between the reported uncertainty and the standard deviation, σrep, and the single grid cell precision, σ, is 1.16.
The probability that the 0.5 ppm accuracy requirement is met is 77%. The probability that the 0.5 ppm/year stability requirement is met is 97%. Overall, this product has therefore reasonable accuracy and high stability.
Summary product quality XCH4_OBS4MIPS v4.6:
The TCCON-validation of the XCH4_OBS4MIPS v4.6 product can be summarized as follows. There is only a small overall average bias of -6.29 ppb. Regional biases estimated from the site-to-site bias variability amount to 5.86 ppb. The average seasonal and spatiotemporal bias amounts to 2.18 ppb and 6.25 ppb, respectively. The overall linear drift of 0.32 ppb/year is much smaller than its site-to-site variability of 0.87 ppb/year. The uncertainty ratio, i.e. the ratio between the reported uncertainty and the standard deviation, σrep, and the single grid cell precision, σ, is 1.29.
The probability that the 10 ppb accuracy requirement is met is 84%. The probability that the 3 ppb/year stability requirement is met is 97%. Overall, this product has therefore very good accuracy and high stability.
1. Product validation methodology
The validation method as used for version 4.6 products XCO2_OBS4MIPS and XCH4_OBS4MIPS is based on comparisons of the XCO2 and XCH4 satellite-derived data products with corresponding XCO2 and XCH4 retrieval from the Total Carbon Column Observing Network (TCCON, http://www.tccon.caltech.edu/) (Wunch et al., 2010, 2011, 2015, Laughner et al., 2024 ).
Stochastic and (quasi-)systematic errors influence the usefulness of our data very differently. Stochastic errors average out quickly in global surface flux inversions, but large-scale (spatial and temporal) systematic errors, such as false inter-hemispheric gradients of a few tenths of a ppm, can significantly degrade the results. The opposite is true for smaller-scale applications. Due to the smaller scale, fewer data points are involved, making stochastic errors more important. However, an application using only data within Europe is not influenced by biased results in Australia. Therefore, we always aim at separating stochastic error components from systematic components. From the stochastic part we infer the precision or uncertainty. The systematic part is analysed in order to conclude on the accuracy. However, we split this term into four: the overall bias or offset, a regionally varying component, a seasonally varying component, and a linear drift. The first term is practically irrelevant, because, basically all applications only analyse gradients. However, the regionally or temporally varying parts of the systematic error are critical as mentioned above. As a measure for the variability of such biases, we compute the standard deviation of the regional and temporal part of the systematic error. In order to address its regional or spatial character, we call the corresponding accuracy measure “relative systematic error” or “spatio-temporal bias”.
1.1. Description of reference data used for validation
1.1.1. TCCON network
For validation of satellite XCO2 and XCH4 retrievals the Total Carbon Column Observing Network (TCCON, http://www.tccon.caltech.edu/) has been established (e.g., Wunch et al., 2010, 2011, 2015, Laughner et al., 2024). This network is the core network used for validation of the satellite XCO2 and XCH4 retrievals. Nevertheless, there are also some limitations as explained in Section 1.4.
TCCON provides XCO2 and XCH4 data products as retrieved from ground-based Fourier Transform Infrared (FTIR) observations based on direct sun observations. The TCCON data products can essentially be directly compared with the satellite-derived XCO2 and XCH4 data products have been used for this purpose extensively in the past as shown in many studies and publications (e.g., Reuter et al., 2020, and references given therein).
The uncertainty of the TCCON reference data (see Wunch et al., 2010, but also the discussions of this uncertainty related to the use of TCCON data for the validation of satellite retrievals in Buchwitz et al., 2015, 2016, and Dils et al., 2014) is:
- TCCON uncertainty XCO2: 0.4 ppm (1-sigma)
- TCCON uncertainty XCH4: 4 ppb (1-sigma)
1.1.2. Traceability to standard
As explained in this document, the satellite-derived XCO2 and XCH4 data products have been validated by comparison with TCCON XCO2 and XCH4 data products, which in turn have been calibrated against the World Meteorological Organization (WMO) in situ trace gas measurement scales (see Wunch et al., 2010). This approach ensures that the satellite XCO2 and XCH4 retrievals are linked to the WMO standards for atmospheric CO2 and CH4 measurements.
1.2. Description of product validation methodology
In this section, we describe the method used to validate the gridded OBS4MIPS XCO2 and XCH4 L3 products. It is based on the validation method applied to precursor C3S L2 XCO2 and XCH4 data sets, but there are some important differences. The method has been adapted to validate L3 products instead of L2 products.
The OBS4MIPS XCO2 and XCH4 data products are validated with TCCON (Wunch et al., 2011) version GGG2020 (Laughner et al., 2024) measurements in a similar way (Reuter et al. 2011). For all comparisons, averaging kernels have been applied and the influence of the smoothing error reduced (Wunch et al., 2011). The TCCON data was gridded to the OBS4MIPS spatio-temporal grid. To ensure that the TCCON measurements used for averaging are representative for the monthly mean, only grid cells with more than 100 TCCON measurements performed on 10 or more different days are taken into account. Co-locations refer to grid cells having both OBS4MIPS and TCCON data. The validation period ranges from 01/2009 to 12/2023 for CO2 and 04/2010 to 12/2023 for CH4. We only consider, for individual stations, time series that cover more than 12 months so that the time series are long enough to ensure robust results. Additionally, it shall be noted that Figure 5 and Figure 8 show that all time series cover at least a period of about 3.5 years resulting in robust results.
For each data product several quality-related quantities have been determined as described in the following.
1.2.1. Per station statistics
For each TCCON station in the gridded TCCON data, the following performance statistics are calculated: number of co- locations, station bias (also known as regional bias), seasonal bias, linear drift, and single grid cell precision.
The main validation results are computed by fitting the following bias model to the difference between the satellite retrievals and the TCCON measurements at each TCCON site.
\[ \Delta X = a_0 + a_1 t + a_2 \sin (2 \pi t + a_3) + \epsilon. \qquad\qquad\qquad\qquad\text{(1)} \]Here, \( \Delta X \) represents the difference (satellite minus TCCON), \( t \) the time, in units of years, of the collocation between TCCON and the satellite measurements, \( \epsilon \) the residual, and \( a_{0-3} \) the free fit parameters. Specifically, \( a_0 \) represents an offset, \( a_1 \) the linear drift and \( a_2 \) the amplitude of the seasonal bias at a TCCON site.
The station or regional bias, \( \Delta_{reg} \) , is computed from the average over time of the fit model:
\[ \Delta_{reg} = \overline{a_0 + a_1 t + a_2 \sin (2 \pi t + a_3)}, \qquad\qquad\qquad\qquad\text{(2)} \]where the overline represents the mean.
The seasonal bias, \( \Delta_{sea} \) , is computed from the standard deviation (denoted as std) of the seasonal term:
\[ \Delta_{sea} = std (a_2 \sin (2 \pi t + a_3) ). \qquad\qquad\qquad\qquad\text{(3)} \]The spatiotemporal bias is computed from the regional and seasonal biases as
\[ \Delta_{spt} = \sqrt{ \Delta_{reg}^2 + \Delta_{sea}^2} \qquad\qquad\qquad\qquad\text{(4)} \]The linear drift, \( \Delta_{dri} \) corresponds to the fit parameter \( a_1, \) , i.e. \( \Delta_{dri} = a_1. \)
The single grid cell precision, \( \sigma \) , is computed from the standard deviation of the residual:
\[ \sigma = std(\epsilon) = std (\Delta X - a_0 - a_1 t - a_2 \sin (2 \pi t + a_3)) \qquad\qquad\qquad\qquad\text{(5)} \]The OBS4MIPS products contain an uncertainty estimate for each grid cell (see ATBD XGHG, 2024). This uncertainty is meant to be the statistical uncertainty (1-sigma) associated with that grid cell value. We compute an average reported uncertainty, \( \sigma_{rep} \) , per station, as the quadratic average over time of the OBS4MIPS reported uncertainty, \( \sigma'_{rep} \) , for each grid cell.
\[ \sigma_{rep} = \sqrt{\overline{\sigma'_{rep}^2}} \qquad\qquad\qquad\qquad\text{(6)} \]1.2.2. Summarizing statistics
Based on the per station statistics, the following summarizing statistics are calculated.
The average station bias and the station-to-station bias variability (which we consider a measure for the regional biases), are computed as the mean and standard deviation of the individual station biases:
\( \overline{ \Delta_{reg}} \pm std(\Delta_{reg}). \)
The average seasonal bias is computed as the mean of the seasonal biases of all TCCON sites:
\( \overline{ \Delta_{sea} }. \)
The overall spatio-temporal bias, \( \overline{ \Delta_{spt} } \) , is computed as:
\[ \overline{ \Delta_{spt} } = \sqrt{ std^2 ( \Delta_{reg} ) + \overline{ \Delta_{sea} }^2 }. \qquad\qquad\qquad\qquad\text{(7)} \]
The average linear drift and the drift uncertainty is computed by:
\( \overline{ \Delta_{dri} } \pm std (\Delta_{dri}). \)
As the linear drift can be assumed to be globally constant, the station-to-station standard deviation of the linear drift can be considered a measure of its uncertainty.
The overall single grid cell precision is computed by:
\[ \overline{ \sigma } = \sqrt{ \overline{ \sigma^2}}, \qquad\qquad\qquad\qquad\text{(8)} \]and the average reported uncertainty is given by:
\[ \overline{ \sigma_{rep} } = \sqrt{ \overline{ \sigma_{rep}^2}}, \qquad\qquad\qquad\qquad\text{(9)} \]To assess the quality of the OBS4MIPS uncertainty estimates, the overall single grid cell precision, \( \overline{\sigma} \) , is compared to the average reported uncertainty, \( \overline{\sigma_{rep}} \) . For that purpose, we computed the uncertainty ratio, defined as:
\[ r = \frac{\overline{\sigma}}{\overline{\sigma_{rep}}}. \qquad\qquad\qquad\qquad\text{(10)} \]If the reported uncertainty is realistic, it should approximately agree with the average precision estimated from the comparison with TCCON. In this case the uncertainty ratio r should be approximately unity. Therefore, the uncertainty ratio r can be used to estimate if the reported uncertainty is realistic.
1.3. Methods for comparison of the achieved performance with the user requirements
The results obtained with the quantitative assessment methods described in Section 1.2.1 and Section 1.2.2 are compared with the Target Requirements (TRs) as given in the Target Requirement and Gap Analysis Document (TR-GAD) (TR-GAD GHG, 2024).
The user requirements are listed in the Target Requirement and Gap Analysis Document (TR-GAD GHG, 2024). They are based on requirements as formulated in documents GCOS-154, GCOS-195, GCOS-200, GCOS-245 and CMUG-RBD, 2012.
The TR-GAD GHG, 2024, document contains explicit requirements for random errors, systematic errors and stability of the Level 2 XCO2 and XCH4 data products in terms of goal (G), breakthrough (B)and threshold (T) requirements. Explicit requirements for Level 3 products are not formulated in TR-GAD GHG, 2024. Instead, it is assumed that the accuracy and stability requirements are also valid for Level 3 (i.e., spatio-temporally averaged) data products.
As explained in Section 2 of TR-GAD GHG, 2024, the GCOS requirements as formulated in GCOS-245 are not applicable for the data products as presented in this document as these new GCOS requirements are formulated for future missions (e.g., CO2M) and are not appropriate for existing satellite sensors that are used for this project. The following is written in TR-GAD GHG, 2024: “Because these new requirements are for future missions, we use in this document (wherever possible) the requirements as have been formulated by the Climate Research Group (CRG) of the GHG-CCI project of ESA’s Climate Change Initiative. We use the latest version, which is the User Requirements Document (URD) referred to as ESA-CCI-GHG-URD, 2024.
To obtain a statement if a certain TR is met or not - or if it is "partially met" - several uncertainties are considered:
- The uncertainty of the estimated parameter (e.g., the uncertainties of the obtained values for "accuracy" and "stability").
- The uncertainty of the reference data (here: TCCON) (if not already included in the uncertainty of the obtained values for "accuracy" and "stability").
- The uncertainty of the comparison method (e.g., considering imperfect collocation of the satellite data and the reference data) (if not already included in the uncertainty of the obtained values for "accuracy" and "stability").
The following discussion is limited to "accuracy" and "stability" as these are the most critical / important data quality figures of merit (FoM) and because the TRs have been defined for them.
The TRs are the following (see also TR-GAD GHG, 2024):
- (Relative) Accuracy XCO2: < 0.5 ppm (1-sigma)
- Stability XCO2: < 0.5 ppm/year
- (Relative) Accuracy XCH4: < 10 ppb (1-sigma)
- Stability XCH4: < 3 ppb/year
1.3.1. (Relative) Accuracy
The term "accuracy" as used here means "relative accuracy" or "relative bias". For many applications (e.g., trend assessments or studies related to regional enhancements) a possible global offset is not critical but spatio-temporal biases are highly relevant. For example, for trend assessments the relative accuracy between different time periods is important (i.e., temporal biases would be critical) and for studies related to regional enhancements the spatial difference is important (i.e., spatial biases would be critical). Therefore, we focus in our performance assessments on “relative accuracy” rather than “absolute accuracy” (see also TRD-GAD GHG, 2024). Nevertheless, the global offset (a single number per product) has been determined and is reported in this document (and can be considered by the users if needed).
The relative accuracy is estimated from mean biases as obtained at the various TCCON validation sites as explained in detail in Section 1.2. The estimated achieved accuracy is in the following referred to as “ACC”. To compare this estimated achieved accuracy ACC with the TR requirement, one cannot simply compare the 2 numbers (by computing achieved minus required accuracy) but has to consider the uncertainty of the accuracy estimate. This uncertainty comes from the uncertainty of the reference data (here TCCON) and the uncertainty of the comparison method (e.g., colocation method and its representativity error). In the following the uncertainty of the accuracy estimate is referred to as “UNC_ACC” and we explain how it is computed and how it is used to estimate the probability that the accuracy target requirement is met.
The uncertainty of the TCCON reference data (see Wunch et al. (2010), but also the discussions of this uncertainty related to the use of TCCON data for the validation of satellite retrievals in Buchwitz et al., 2015, 2016, and Dils et al., 2014) is:
- TCCON uncertainty XCO2: 0.4 ppm (1-sigma)
- TCCON uncertainty XCH4: 4 ppb (1-sigma)
These uncertainties are increased by 50% to also consider other error sources, especially error of the comparison method such as the representativity error. The assumed uncertainty ("UNC_ACC") of the estimated accuracy value ("ACC") are therefore:
- UNC_ACC XCO2: 0.6 ppm (1-sigma)
- UNC_ACC XCH4: 6 ppb (1-sigma)
In summary, we now have ACC +/- UNC_ACC (1-sigma) for the estimated relative accuracy or spatio-temporal bias and its uncertainty. These values are interpreted as the mean and the standard deviation of an underlying probability density function (pdf).
ACC is a non-negative number and the Target Requirement (TR) for accuracy defines an “acceptable range” or interval of “acceptable” accuracy values as [0, TR[, i.e., in order to meet the requirements ACC shall be smaller than TR but will be larger than (or equal to) zero. Because of this non-negativity, ACC cannot be distributed according to a Gaussian (“normal”) probability density function (pdf) (esp. if the mean is much smaller than the standard deviation) but it may be reasonable to assume that the overall distribution is a lognormal distribution (last accessed 15/11/2023), with parameters selected such that the lognormal pdf is very similar to a Gaussian pdf if the mean is on the order or larger than the standard deviation.
The probability density function (pdf) of the lognormal distribution is:
\[ y = f(x|\mu, \sigma) = \frac{1}{x \sigma \sqrt{2 \pi}} \exp\left(- \frac{(\log x - \mu)^2}{2 \sigma^2}\right), \quad \text{for} \ x \ > \ 0. \qquad\qquad\qquad\qquad\text{(11)} \]The lognormal distribution has parameters µ and σ, which are related to parameters mean m = ACC and variance ν = UNC_ACC2 as follows:
\[ \mu = \log \left( \frac{m^2}{\sqrt{\nu + m^2}} \right), \qquad\qquad\qquad\qquad\text{(12)} \] \[ \sigma = \sqrt{\log \left( \frac{\nu}{m^2} + 1 \right)}. \qquad\qquad\qquad\qquad\text{(13)} \]The cumulative distribution function (cdf) of the lognormal distribution is:
\[ P = F(x|\mu, \sigma) = \frac{1}{\sigma \sqrt{2 \pi}} \int_0^x \frac {1}{t} \exp\left(- \frac{(\log t - \mu)^2}{2 \sigma^2}\right) \text{d}t, \quad \text{for} \ x \ > \ 0. \qquad\qquad\qquad\qquad\text{(14)} \]This function is used to compute the probability, that the accuracy requirement is met, see Figure 1 for XCO2 and Figure 2 for XCH4.
Figure 1: Probability that the XCO2 accuracy TR is met as a function of the achieved accuracy.
Figure 2: Probability that the XCH4 accuracy TR is met as a function of the achieved accuracy.
1.3.2. Stability
For the TR assessment, the stability assessment is limited to the linear bias trend / drift.
As for the accuracy assessment, it is assumed that the value for stability has been obtained assuming error free TCCON observations and an error free comparison method. In contrast to the accuracy assessment approach, it is assumed that the uncertainty of the stability value is known (it corresponds to the (1-sigma) slope (SLO) error of the linear fit). The result of the stability assessment is: STA +/- UNC_SLO.
To consider the uncertainty of the reference data we assume that the TCCON data approximately meet the following stability requirements:
- XCO2 stability: 0.2 ppm/year
- XCH4 stability: 1 ppb/year
These uncertainties need to be added quadratically (via Root-Sum-Square (RSS)) to UNC_SLO to obtain the overall uncertainty UNC_STA.
As shown in Table S-1 for XCO2 and Table S-2 for XCH4 in column "Long-term drift" in document PVIRv5 (Buchwitz et al., 2017) typical values for STA +/- UNC_SLO are (if the uncertainty is converted to 1-sigma):
- XCO2: +0.1 +/- 0.07 (1-sigma) ppm/year
- XCH4: -0.8 +/- 0.4 (1-sigma) ppb/year
These values are listed here only for illustration (the exact value depends on product and assessment method).
Quadratically adding the assumed TCCON uncertainty gives for this example for STA +/- UNC_STA:
- XCO2: +0.1 +/- 0.21 (1-sigma) ppm/year
- XCH4: -0.8 +/- 1.08 (1-sigma) ppb/year
In contrast to ACC, STA can also be negative and we use a Gaussian probability density function N(x, mean=STA, sigma=UNC_STA) to compute the probability that the stability TR is met. This probability is the integral of N over the interval as defined by the stability TR requirement, i.e., interval [-TR, +TR], or simply the difference between two different values of the cumulative distribution function Nc(x, mean=STA, sigma=UNC_STA) (namely at x=TR and x=-TR). The probability P that the stability TR is met for XCO2 for a given value of STA is therefore for this example: P(STA) = Nc (+0.5, mean=+0.1, sigma=0.2) – Nc(-0.5, mean=+0.1, sigma=0.2) = 97%. This means that in these cases it is almost certain that the stability TR is met.
Figure 3 shows typical probability functions.
Figure 3: Probability functions used to obtain the probability that the stability requirement is met. Top: for XCO2. Bottom: for XCH4.
1.4. Known limitations
The TCCON network (see also the TCCON wiki pages for the TCCON Sites and for the Sites Pages) consists of the sites as shown in Figure 4. As can be seen, it is relatively dense in the USA, in Europe and in Japan but overall, the TCCON network is relatively sparse (e.g., no or only very few sites in Russia, South America and Africa) and does not cover all conditions, which affect or can affect the quality of the satellite XCO2 and XCH4 retrievals (e.g., deserts due to their high surface albedo combined with potentially high amounts of specific aerosol types such as desert dust storm mineral aerosols). Typically, the TCCON network underrepresents desert-like surfaces, open oceans, the tropics and the southern hemisphere in general.
Due to the large grid boxes of the OBS4MIPS XCO2 and XCH4 products, the TCCON data might sometimes not be representative for the real average XGHG in that grid box, specially in cases where the TCCON station is close to the boundaries of the grid box. One could speculate that this contributes to the relatively large XCH4 bias seen in the validation results (see Section 2.2) at the TCCON site Nicosia, which is located in Cypress at the latitude 35.141ºN. As the OBS4MIPS XCH4 L3 product is gridded 5°x5°, the TCCON site nearly lies on a grid box boundary which is the worst case because it becomes less representative for the grid box average.
It should also be noted that some bias correction methods of the individual L2 products contributing to EMMA use TCCON data, which can reduce the independence between the satellite product and the validation data set to some degree.
The TCCON project started in 2004 and the number of operational sites has increased over time. The TCCON Network provides temporally dense observations but sometimes significantly after acquisition.
Figure 4: Location of TCCON sites. Source: https://tccondata.org/.
The TCCON network is the core network for the validation of the satellite XCO2 and XCH4 retrievals and is therefore essential for this part of the C3S service.
It would therefore be highly beneficial for this service:
- if the TCCON network were expanded to better cover all geophysical conditions relevant for the quality assessment of the satellite retrievals.
- if the TCCON XCO2 and XCH4 retrievals were available faster.
As a minimum, it needs to be guaranteed that the existing network remains in place, but unfortunately even this is currently not guaranteed.
2. Validation results
In this section, detailed validation results are shown. The validation results for the Level 3 XCO2 product are presented in Section 2.1 and the XCH4 validation results are shown in Section 2.2. For each data product a set of well defined FoMs is computed to summarize the validation results.
The validation is based on comparisons with Total Carbon Column Observing Network (TCCON) XCO2 and XCH4 retrievals. For the results presented in this document, the TCCON station IDs as shown in Table 1 have been used to name each corresponding TCCON station.
Table 1: Station IDs corresponding to each TCCON Station name.
Station ID | Station name |
|---|---|
so | Sodankylä |
et | East Trout Lake |
bi | Bialystok |
pa | Park Falls |
gm | Garmisch-Partenkirchen |
or+pr | Orleans + Paris |
ka | Karlsruhe |
rj | Rikubetsu |
xh | Xianghe |
oc | Lamont |
ni | Nicosia |
tk | Tsukuba_tk |
js | Saga |
hf | Hefei |
ci+df+jc+jf | Pasadena_ci + Edwards + Pasadena_jc + Pasadena_jf |
iz | Izana |
bu | Burgos |
db | Darwin |
ra | Reunion Island |
wg | Wollongong |
lh+ll+lr | Lauder_lh + Lauder_lh + Lauder_lr |
2.1. Validation results for Level 3 OBS4MIPS XCO2 product
Figure 5 shows all 1387 co-located OBS4MIPS and TCCON XCO2 gridded results used for the validation study. One can see that the temporal sampling differs from site to site and that the L3 OBS4MIPS XCO2 product captures the year-to-year increase and the seasonal features well
The overall agreement of the OBS4MIPS data with TCCON data at all sites is illustrated in Figure 6. The histogram of the difference OBS4MIPS – TCCON (Figure 6, top) shows a near Gaussian distribution with a center at 0.31 ppm and a standard deviation of 1.03 ppm. The OBS4MIPS vs. TCCON plot (Figure 6, bottom) shows a pronounced clustering along the one-to-one line. This is supported by a good agreement of the orthogonal distance regression with the one to-one line and high Pearson correlation coefficient of 0.994. These results provide a first rough overview of OBS4MIPS 's agreement with TCCON.
The results of all site performance statistics as well as the summarizing performance statistics are illustrated in Figure 7 and listed in Table 2. The Product Quality Summary Table for the OBS4MIPS XCO2 product is shown as Table 3.
Figure 5: Co-located OBS4MIPS and TCCON XCO2 results used for the validation study.
Figure 6: Overall overview on the agreement of the OBS4MIPS data with TCCON data at all sites. Top: Normalized histograms of the difference OBS4MIPs minus TCCON. Bottom: OBS4MIPS vs. TCCON including one-to-one line, orthogonal distance regression (ODR), and Pearson correlation coefficient 𝛿.
Figure 7: Validation results for XCO2 OBS4MIPS monthly averages. From left to right, the figure shows the per site performance statistics (Section 1.2.1) regional. \( (\Delta_{reg}), \) seasonal \( (\Delta_{sea}), \) and spatiotemporal bias \( (\Delta_{spt}), \) the linear drift \( (\Delta_{dri}), \) the actual \( (\sigma) \) and reported precision \( (\sigma_{rep}) \) and the number of soundings ( # ). TCCON sites are ordered from top to bottom by average latitude of the co-located satellite soundings. The last row includes the summarizing performance statistics as defined in Section 1.2.2.
Table 2: Validation results for XCO2 OBS4MIPS. From left to right, the table shows the per site performance statistics (Section 1.2.1) regional. \( (\Delta_{reg}), \) seasonal \( (\Delta_{sea}), \) and spatiotemporal bias \( (\Delta_{spt}), \) the linear drift \( (\Delta_{dri}), \) the actual \( (\sigma) \) and reported precision \( (\sigma_{rep}) \) and the number of soundings ( # ). TCCON sites are order from top to bottom by average latitude of the co-located satellite soundings. The last row includes the summarizing performance statistics as defined in Section 1.2.2.
Station | Δreg [ppm] | Δsea [ppm] | Δspt [ppm] | Δdri [ppm/year] | σ [ppm] | σrep [ppm] | # |
|---|---|---|---|---|---|---|---|
so | 0.11 | 0.07 | 0.13 | 0.00 | 0.90 | 1.18 | 36 |
et | 0.68 | 0.25 | 0.72 | -0.10 | 1.32 | 1.40 | 41 |
bi | 0.29 | 0.41 | 0.50 | -0.07 | 0.96 | 0.98 | 50 |
pa | 0.26 | 0.44 | 0.51 | -0.03 | 1.17 | 1.05 | 125 |
gm | 0.66 | 0.22 | 0.70 | 0.05 | 0.85 | 1.09 | 79 |
or+pr | 0.22 | 0.35 | 0.42 | 0.06 | 0.92 | 0.92 | 88 |
ka | 0.17 | 0.39 | 0.42 | 0.10 | 1.02 | 0.96 | 45 |
rj | 0.59 | 0.03 | 0.59 | -0.02 | 0.58 | 1.30 | 20 |
xh | 1.18 | 0.10 | 1.18 | 0.11 | 1.08 | 1.23 | 54 |
oc | 0.11 | 0.26 | 0.28 | 0.03 | 0.55 | 0.77 | 145 |
ni | 0.11 | 0.13 | 0.17 | 0.24 | 0.89 | 0.92 | 38 |
tk | 0.26 | 0.39 | 0.47 | 0.10 | 0.87 | 1.11 | 49 |
js | 0.75 | 0.11 | 0.76 | 0.18 | 1.05 | 1.26 | 75 |
hf | 0.36 | 0.77 | 0.85 | 0.01 | 1.33 | 1.27 | 25 |
ci+df+jc+jf | 0.25 | 0.21 | 0.33 | 0.06 | 0.82 | 0.97 | 157 |
iz | 0.10 | 0.39 | 0.40 | 0.04 | 0.71 | 1.11 | 42 |
bu | 0.05 | 0.14 | 0.15 | 0.23 | 0.82 | 0.92 | 35 |
db | -0.21 | 0.26 | 0.33 | -0.16 | 0.65 | 0.81 | 86 |
ra | 0.41 | 0.16 | 0.44 | -0.22 | 0.38 | 0.55 | 23 |
wg | 0.36 | 0.08 | 0.37 | -0.06 | 0.75 | 0.96 | 107 |
lh+ll+lr | 0.34 | 0.38 | 0.51 | -0.17 | 0.91 | 1.17 | 67 |
Summary | 0.34 ± 0.30 | 0.26 | 0.40 | 0.02 ± 0.12 | 0.91 | 1.06 | 1387 |
The validation results show that there is only a small overall average station bias of 0.34 ppm. Regional biases estimated from the site-to-site bias variability amount to 0.30 ppm and are strongly influenced by the relatively large bias of 1.18 ppm at the TCCON site Xianghe (xh). The average seasonal and spatiotemporal bias amounts to 0.26 ppm and 0.40 ppm, respectively. The overall linear drift of 0.02 ppm/year is much smaller than its site-to-site variability of 0.12 ppm/year and, therefore, considered not significant. The uncertainty ratio, i.e. the ratio between the single grid cell precision, \( \sigma, \) and the reported uncertainty, \( \sigma_{rep}, \) is 1.16, which is close to unity, indicating good agreement between the OBS4MIPS reported uncertainty and the overall precision.
Table 3: Product Quality Summary Table for product XCO2_OBS4MIPS.
Product Quality Summary Table for Product: XCO2_OBS4MIPS | ||||
|---|---|---|---|---|
Parameter [unit] | Achieved performance | Requirement | TR | Comments |
Overall uncertainty [ppm] | 0.91 | - | - | No requirement but small value expected for a high quality data product. |
Mean bias [ppm] | 0.34 | - | - | No requirement but value close to zero expected for a high quality data product. |
Accuracy: Relative systematic error [ppm] | Spatio-temporal bias: 0.4 (1-sigma) | < 0.5 | Probability that accuracy TR is met: 77% | - |
Stability: Linear bias trend [ppm/year] | 0.02 +/- 0.12 (1-sigma) | < 0.5 | Probability that stability TR is met: 97% | - |
The probability that the 0.5 ppm accuracy requirement is met is 77%.
The probability that the 0.5 ppm/year stability requirement is met is 97%.
Overall, this product has therefore reasonable accuracy and high stability.
2.2. Validation results for Level 3 OBS4MIPS XCH4 product
Figure 8 shows all 1495 co-located OBS4MIPS and TCCON XCH4 gridded results used for the validation study. One can see that the temporal sampling differs from site to site and that the L3 OBS4MIPS XCH4 product captures the year-to-year increase and the seasonal features well.
The overall agreement of the OBS4MIPS data with TCCON data at all sites is illustrated in Figure 9. The histogram of the difference OBS4MIPS – TCCON (Figure 9, top) shows a near Gaussian distribution with a center at -6.70 ppm and a standard deviation of 8.16 ppm. The OBS4MIPS vs. TCCON plot (Figure 9, bottom) shows a pronounced clustering along the one-to-one line. This is supported by a good agreement of the orthogonal distance regression with the one to-one line and high Pearson correlation coefficient of 0.985. These results provide a first rough overview of OBS4MIPS 's agreement with TCCON.
The results of all site performance statistics as well as the summarizing performance statistics are illustrated in Figure 10 and listed in Table 4.
The Product Quality Summary Table for the OBS4MIPS XCH4 product is shown as Table 5.
Figure 8: Co-located OBS4MIPS and TCCON XCH4 results used for the validation study.
Figure 9: Overall overview on the agreement of the XCH4 OBS4MIPS data with TCCON data at all sites. Top: Normalized histograms of the difference OBS4MIPS minus TCCON. Bottom: OBS4MIPS vs TCCON including one-to-one line, orthogonal distance regression (ODR), and Pearson correlation coefficient 𝛿.
Figure 10: Validation results for OBS4MIPS monthly averages. From left to right, the figure shows the per site performance statistics (Section 1.2.1) regional.
\( (\Delta_{reg}), \)
seasonal
\( (\Delta_{sea}), \)
and spatiotemporal bias
\( (\Delta_{spt}), \)
the linear drift
\( (\Delta_{dri}), \)
the actual
\( (\sigma) \)
and reported precision
\( (\sigma_{rep}) \)
and the number of soundings ( # ). TCCON sites are ordered from top to bottom by average latitude of the co-located satellite soundings. The last row includes the summarizing performance statistics as defined in Section 1.2.2.
Table 4: Validation results for XCH4 OBS4MIPS. From left to right, the table shows the per site performance statistics (Section 1.2.1) regional. \( (\Delta_{reg}), \) seasonal \( (\Delta_{sea}), \) and spatiotemporal bias \( (\Delta_{spt}), \) the linear drift \( (\Delta_{dri}), \) the actual \( (\sigma) \) and reported precision \( (\sigma_{rep}) \) and the number of soundings ( # ). TCCON sites are order from top to bottom by average latitude of the co-located satellite soundings. The last row includes the summarizing performance statistics as defined in Section 1.2.2.
Station | Δreg [ppb] | Δsea [ppb] | Δspt [ppb] | Δdri [ppb/year] | σ [ppb] | σrep [ppb] | # |
|---|---|---|---|---|---|---|---|
so | -4.79 | 1.44 | 5.00 | 0.20 | 8.07 | 9.29 | 55 |
et | -3.14 | 3.40 | 4.62 | -0.07 | 4.90 | 9.63 | 51 |
bi | -5.30 | 1.48 | 5.50 | -0.41 | 5.84 | 6.11 | 54 |
pa | -3.75 | 2.39 | 4.45 | -0.01 | 6.85 | 8.30 | 136 |
gm | -0.40 | 2.35 | 2.39 | 0.16 | 5.91 | 6.92 | 78 |
or+pr | -6.15 | 1.10 | 6.24 | 0.07 | 5.41 | 6.84 | 102 |
ka | -9.38 | 0.49 | 9.39 | -0.25 | 5.23 | 6.86 | 48 |
rj | 1.83 | 3.70 | 4.13 | -0.25 | 9.33 | 10.08 | 26 |
xh | -5.54 | 5.09 | 7.52 | 2.63 | 6.61 | 8.46 | 54 |
oc | -7.11 | 1.05 | 7.19 | 0.04 | 3.82 | 5.03 | 145 |
ni | -17.32 | 3.12 | 17.60 | 2.00 | 5.42 | 8.56 | 35 |
tk | -9.45 | 1.43 | 9.55 | 0.69 | 6.00 | 7.28 | 50 |
js | -7.70 | 0.88 | 7.75 | 0.81 | 6.36 | 7.66 | 77 |
hf | -7.65 | 6.04 | 9.75 | 0.95 | 7.68 | 8.48 | 21 |
ci+df+jc+jf | -10.73 | 1.50 | 10.83 | 1.12 | 5.25 | 9.34 | 157 |
iz | 12.45 | 1.96 | 12.61 | 1.08 | 4.93 | 7.88 | 43 |
bu | -7.35 | 2.64 | 7.81 | -0.86 | 6.17 | 6.86 | 38 |
db | -13.32 | 1.71 | 13.43 | -0.51 | 3.79 | 5.33 | 86 |
ra | -11.44 | 1.20 | 11.50 | -0.98 | 3.81 | 5.37 | 28 |
wg | -8.34 | 2.34 | 8.66 | 0.50 | 5.54 | 7.44 | 108 |
lh+ll+lr | -7.57 | 0.44 | 7.58 | -0.10 | 6.97 | 9.44 | 103 |
Summary | -6.29 ± 5.86 | 2.18 | 6.25 | 0.32 ± 0.87 | 6.06 | 7.81 | 1495 |
The validation results show that there is only a small overall average station bias of -6.29 ppb. Regional biases estimated from the site-to-site bias variability amount to 5.86 ppb. The average seasonal and spatio-temporal bias amounts to 2.18 ppb and 6.25 ppb, respectively. The overall linear drift of 0.32 ppb/year is much smaller than its site-to-site variability of 0.87 ppb/year and, therefore, considered not significant.
A drift in the satellite data product would most likely affect all sites. Therefore, we consider the station-to-station drift variability a measure of drift uncertainty. For example, it is unclear what causes the drift at Pasadena+Edwards (ci+df+jc+jf), especially because we do not observe a comparable drift at the nearby TCCON site Lamont (oc).
The uncertainty ratio, the ratio between the single grid cell precision, \( \sigma, \)
and the reported uncertainty, \( \sigma_{rep}, \) is 1.29, which is close to unity, indicating good agreement between the OBS4MIPS reported uncertainty and the overall precision.
Table 5: Product Quality Summary Table for product XCH4_OBS4MIPS.
Product Quality Summary Table for Product: XCH4_OBS4MIPS | ||||
|---|---|---|---|---|
Parameter [unit] | Achieved performance | Requirement | TR | Comments |
Overall uncertainty [ppb] | 6.1 | - | - | No requirement but small value expected for a high quality data product. |
Mean bias [ppb] | -6.3 | - | - | No requirement but value close to zero expected for a high quality data product. |
Accuracy: Relative systematic error [ppb] | Spatio-temporal bias: 6.25 (1-sigma) | < 10 | Probability that accuracy TR is met: 84% | - |
Stability: Linear bias trend [ppb/year] | 0.32 +/- 0.87 (1-sigma) | < 3 | Probability that stability TR is met: 97% | - |
The probability that the 10 ppb accuracy requirement is met is 84%.
The probability that the 3 ppb/year stability requirement is met is 97%.
Overall, this product has therefore very good accuracy and high stability.
3. Climate Change Assessment
The OBS4MIPS products analysed in this PQAR are merged products that were generated using various retrieval algorithms and satellite instruments (see Figures 6 and 7 in ATBD XGHG, 2024). Although the data were homogenized (ATBD XGHG, 2024), there may still be differences between the data products used, e.g. in sampling and data quality. This can lead to temporal or spatial changes in the characteristics of the OBS4MIPS products, which are, however, reflected in the estimated uncertainties.
In climate applications, it is important that the time series of a parameter studied are significantly longer than the changes caused by its natural variability. For this reason, climate scientists usually analyse at least several decades. The data analysed in this PQAR span more than two decades and the user requirements for stability are also met.
The OBS4MIPS data were thoroughly validated with co-located TCCON data in this PQAR. However, due to the spatially sparse sampling of TCCON sites (Figure 4), some geophysical conditions are underrepresented in the validation. These include, for example, deserts, some tropical regions and the southern hemisphere in general. Figure 2 and Figure 3 of PUGS XGHG, 2024 shows that the uncertainty (mainly due to the scatter between algorithms) is the highest in some of these regions.
In the past, predecessor versions of the OBS4MIPS data have already been used for climate applications such as the annual Copernicus ESTOC (European State of the Climate) Greenhouse Gas Concentrations report (ESTOC GHG, 2023). These data products were used to compute global average annual atmospheric concentrations and their year-to-year increase for both CO2 and CH4.
4. Application(s) specific assessments
Previous versions of the OBS4MIPS XCO2 and products has also been used in the peer-reviewed publications of Lauer et al. (2017) and Gier et al. (2020). In these publications, the OBS4MIPS XCO2 products were used to evaluate the performance of CMIP (Coupled Model Intercomparison Project) models in its Phase 5 (CMIP5) in Lauer et al. (2017), and both its Phases 5 and 6 (CMIP5 and CMIP6) in Gier et al. (2020).
5. Compliance with user requirements concerning data quality
This section summarizes the achieved data quality including comparisons with the required data quality.
The development of satellite-derived gridded ECV products in OBS4MIPS format started in the framework of ESA’s Climate Change Initiative (CCI) via the GHG-CCI project (last access: 3-Mar-2025) (see also Reuter et al., 2020). The envisaged main application is comparison with climate models. The GHG-CCI project team therefore proposed already several years ago to generate XCO2 and XCH4 products in OBS4MIPS format at the described spatio-temporal resolution. That spatio-temporal resolution was assumed to be appropriate for climate model comparisons considering also the characteristics of existing satellites. It was later confirmed by scientific studies (e.g., Lauer et al., 2017, and Gier et al., 2020) that the generated products were in fact very useful for the envisaged application.
Table 6 compares the required and the achieved performance for random error (precision), required accuracy (in terms of spatio-temporal biases) and stability (in terms of linear bias drift). The data quality level is also summarized in Section 5.1 for XCO2 and Section 5.2 for XCH4 (see Section 1.3 for a more detailed explanation on the reported probabilities).
Table 6: Compliance with User Requirements. XCO2 and XCH4 random (“precision”), systematic error and stability requirements (from TR-GAD GHG, 2024). Abbreviations: G=Goal, B=Breakthrough, T=Threshold requirement. §) Required systematic error after an empirical bias correction that does not use the verification data. #) Required systematic error and stability after bias correction, where bias correction is not limited to the application of a constant offset / scaling factor.
Parameter | Requirement type | Requirement | Reported value | Probability that TR is met | ||
|---|---|---|---|---|---|---|
G | B | T | ||||
CO2 | Random error (precision) (10002 km2 monthly) (ppm) | < 0.3 | < 1.0 | < 1.3 | 0.91 (<= 1.0, within Breakthrough) | - |
Accuracy: Relative systematic error (ppm) | < 0.2 (absolute) | < 0.3 (relative§)) | < 0.5 (relative#)) | 0.4 (<= 0.5, within Threshold) | 77% | |
Stability: Linear bias trend (ppm/year) | < 0.2 (absolute) | < 0.3 (relative§)) | < 0.5 (relative#)) | 0.02 (<= 0.2, within Goal) | 97% | |
CH4 | Random error (precision) (10002 km2 monthly) (ppb) | < 3 | < 5 | < 11 | 6.1 (<= 11, within Threshold) | - |
Accuracy: Relative systematic error (ppb) | < 1 (absolute) | < 5 (relative§)) | < 10 (relative#)) | 6.25 (<= 10, within Threshold) | 84% | |
Stability: Linear bias trend (ppb/year) | < 1 (absolute) | < 2 (relative§)) | < 3 (relative#)) | 0.32 (<=1, within Goal) | 97% | |
5.1. Summary data quality Level 3 XCO2 product
The validation of Level 3 product XCO2_OBS4MIPS can be summarized as follows:
- The overall monthly mean uncertainty is 0.91 ppm and the mean bias is 0.34 ppm. Relative systematic error, i.e., the spatio-temporal bias, is 0.4 ppm (1-sigma). The computed linear drift of 0.02±0.12 ppm/year (1-sigma) is small and not significant.
- The probability that the 0.5 ppm accuracy requirement is met is 77%.
- The probability that the 0.5 ppm/year stability requirement is met is 97%.
- Overall, this product has therefore reasonable accuracy and high stability.
5.2. Summary data quality Level 3 XCH4 product
The validation of Level 3 product XCH4_OBS4MIPS can be summarized as follows:
- The overall monthly mean uncertainty is 6.1 ppb and the mean bias is -0.63 ppb. Relative systematic error, i.e., the spatio-temporal bias, is 6.25 ppb (1-sigma). The computed linear drift of 0.32±0.87 ppb/year (1-sigma) is small and not significant.
- The probability that the 10 ppb accuracy requirement is met is 84%.
- The probability that the 3 ppb/year stability requirement is met is 97%.
- Overall, this product has therefore very good accuracy and high stability.
Acknowledgements
We acknowledge previous funding by the European Space Agency (ESA) via Climate Change Initiative (CCI) project GHG-CCI. This funding significantly enhanced the quality of the retrieval algorithms and related documentation. This resulted in more mature data products as needed for an operational project such as the Copernicus Climate Change Service (C3S). We also acknowledge the availability of GOSAT and GOSAT-2 data products via the ESA GOSAT Third Party Mission (TPM) archive.
We are also very grateful to the GOSAT/GOSAT-2 teams in Japan comprising the Japan Aerospace Exploration Agency (JAXA), the National Institute for Environmental Studies (NIES), and the Ministry of the Environment (MOE) for providing access to the GOSAT and GOSAT-2 Level 1 and Level 2 data products.
We also acknowledge the availability of OCO-2 Level 1 and Level 2 (XCO2) data products from NASA, which have been used for the generation of the XCO2_OBS4MIPS product. This product also includes OCO-2 XCO2 retrieved at Univ. Bremen with the FOCAL algorithm. The FOCAL activities would not have been possible without funding from University of Bremen, from the EU H2020 projects CHE (grant agreement ID: 776186) and VERIFY (Grant agreement ID: 776810), from ESA via project GHG-CCI+ and from EUMETSAT project FOCAL-CO2M.
The TCCON data were obtained from the TCCON Data Archive hosted by CaltechDATA at https://tccondata.org
References
ATBD MTGHG, 2024: Crevoisier, C., Meilhac, N., C3S Greenhouse Gas (GHG: MTCO2 v10.1 & MTCH4 v10.2): Algorithm Theoretical Basis Document (ATBD)
ATBD XGHG, 2024: Reuter , M., Fuentes Andrade, B., Buchwitz, M., C3S Greenhouse Gas (GHG: CO2 & CH4) v4.6: Algorithm Theoretical Basis Document (ATBD), C3S project C3S2_313a_DLR, 2024. Link: https://confluence.ecmwf.int/x/AS-OIQ
Bojinski et al., 2014: Bojinski, S., M. Verstraete, T.C. Peterson, C. Richter, A. Simmons, M. Zemp, The Concept of Essential Climate Variables in Support of Climate Research, Applications, and Policy. Bull. Amer. Meteor. Soc., 95, 1431–1443. Available at http://journals.ametsoc.org/doi/pdf/10.1175/BAMS-D-13-00047.1, 2014.
Buchwitz et al., 2015: Buchwitz, M., Reuter, M., Schneising, O., Boesch, H., Guerlet, S., Dils, B., Aben, I., Armante, R., Bergamaschi, P., Blumenstock, T., Bovensmann, H., Brunner, D., Buchmann, B., Burrows, J.P., Butz, A., Chédin, A., Chevallier, F., Crevoisier, C.D., Deutscher, N.M., Frankenberg, C., Hase, F., Hasekamp, O.P., Heymann, J., Kaminski, T., Laeng, A., Lichtenberg, G., De Mazière, M., Noël, S., Notholt, J., Orphal, J., Popp, C., Parker, R., Scholze, M., Sussmann, R., Stiller, G.P., Warneke, T., Zehner, C., Bril, A., Crisp, D., Griffith, D.W.T., Kuze, A., O'Dell, C., Oshchepkov, S., Sherlock, V., Suto, H., Wennberg, P., Wunch, D., Yokota, T., Yoshida, Y., The Greenhouse Gas Climate Change Initiative (GHG-CCI): comparison and quality assessment of near-surface-sensitive satellite-derived CO2 and CH4 global data sets. Remote Sens. Environ. 162:344–362, http://dx.doi.org/10.1016/j.rse.2013.04.024, 2015.
Buchwitz et al., 2016: Buchwitz, M., Reuter, M., Schneising, O., Hewson, W., Detmers, R. G., Boesch, H., Hasekamp, O. P., Aben, I., Bovensmann, H., Burrows, J. P., Butz, A., Chevallier, F., Dils, B., Frankenberg, C., Heymann, J., Lichtenberg, G., De Mazière, M., Notholt, J., Parker, R., Warneke, T., Zehner, C., Griffith, D. W. T., Deutscher, N. M., Kuze, A., Suto, H., and Wunch, D.:, Global satellite observations of column-averaged carbon dioxide and methane: The GHG-CCI XCO2 and XCH4 CRDP3 data, Remote Sensing of Environment (in press), Special Issue on Essential Climate Variables, DOI: 10.1016/j.rse.2016.12.027, (link: http://dx.doi.org/10.1016/j.rse.2016.12.027
), 2016.
Buchwitz et al., 2017: ESA Climate Change Initiative (CCI) Product Validation and Intercomparison Report (PVIR) for the Essential Climate Variable (ECV) Greenhouse Gases (GHG) for data set Climate Research Data Package No. 4 (CRDP#4), Version 5.0, 9. Feb. 2017, 2017.
CMUG-RBD, 2012: Climate Modelling User Group Requirements Baseline Document, Deliverable 1.2, Number D1.2, ESA Climate Change Initiative (CCI), Version 2.0, 17 Dec 2012, 2012. Link: https://climate.esa.int/media/documents/CMUG_D1.2_URD_v2.0.pdf
Dils et al., 2014: B. Dils, M. Buchwitz, M. Reuter, O. Schneising, H. Boesch, R. Parker, S. Guerlet, I. Aben, T. Blumenstock, J. P. Burrows, A. Butz, N. M. Deutscher, C. Frankenberg, F. Hase, O. P. Hasekamp, J. Heymann, M. De Mazière, J. Notholt, R. Sussmann, T. Warneke, D. Griffith, V. Sherlock, D. Wunch :The Greenhouse Gas Climate Change Initiative (GHG-CCI): Comparative validation of GHG-CCI SCIAMACHY/ENVISAT and TANSO-FTS/GOSAT CO2 and CH4 retrieval algorithm products with measurements from the TCCON network, Atmos. Meas. Tech., 7, 1723-1744, 2014.
ESA-CCI-GHG-URD, 2024: Chevallier, F., Marshall, J., et al., User Requirements Document (URD) for the Essential Climate Variable (ECV) Greenhouse Gases (GHG), ESA Climate Change Initiative (CCI) GHG-CCI+ project, Version 4.0, 16-May-2024, pp. 40, 2024. Link: https://www.iup.uni-bremen.de/carbon_ghg/docs/GHG-CCIplus/URDv4/URDv4.0_GHG-CCI+.pdf
ESTOC GHG, 2023: European State of the Climate: Greenhouse gas concentrations, compiled by the Copernicus Climate Change Service (C3S) and the World Meteorological Organization (WMO). Link: https://climate.copernicus.eu/esotc/2023/greenhouse-gas-concentrations
GCOS-154: Global Climate Observing System (GCOS): SYSTEMATIC OBSERVATION REQUIREMENTS FOR SATELLITE-BASED DATA PRODUCTS FOR CLIMATE - 2011 Update - Supplemental details to the satellite-based component of the "Implementation Plan for the Global Observing System for Climate in Support of the UNFCCC (2010 Update)", December 2011, prepared by World Meteorological Organization (WMO), Intergovernmental Oceanographic Commission, United Nations Environment Programme (UNEP), International Council for Science, Doc.: GCOS 154, link: http://cci.esa.int/sites/default/files/gcos-154.pdf, 2011.
GCOS-195: Status of the Global Observing System for Climate, GCOS-195, 2015. Link: https://library.wmo.int/doc_num.php?explnum_id=7213
GCOS-200: The Global Observing System for Climate: Implementation Needs, World Meteorological Organization (WMO), GCOS-200 (GOOS-214), pp. 325. Link: http://unfccc.int/files/science/workstreams/systematic_observation/application/pdf/gcos_ip_10oct2016.pdf, 2016.
GCOS-245: The 2022 GCOS ECVs Requirements, WMO, pp. 244, 2022. Link: https://library.wmo.int/doc_num.php?explnum_id=11318
Gier et al., 2020: Gier, B. K., Buchwitz, M., Reuter, M., Cox, P. M., Friedlingstein, P., and Eyring, V.: Spatially resolved evaluation of Earth system models with satellite column-averaged CO2, Biogeosciences, 17, 6115-6144, https://doi.org/10.5194/bg-17-6115-2020, 2020.
Lauer et al., 2017: Lauer, A., Eyring, V., Righi, M., Buchwitz, M., Defourny, P., Evaldsson, M., Friedlingstein, P., de Jeu, R., de Leeuw, G., Loew, A., Merchant, C. J., Mueller, B., Popp, T., Reuter, M., Sandven, S., Senftleben, D., Stengel, M., Van Roozendael, M., Wenzel, S., and Willen, U.: Benchmarking CMIP5modelswith a subset of ESA CCI Phase 2 data using the ESMValTool, Remote Sensing of Environment 203, 9-39, http://dx.doi.org/10.1016/j.rse.2017.01.007, 2017.
Laughner et al., 2024: Laughner, J. L., Toon, G. C., Mendonca, J., Petri, C., Roche, S., Wunch, D., Blavier, J.-F., Griffith, D. W. T., Heikkinen, P., Keeling, R. F., Kiel, M., Kivi, R., Roehl, C. M., Stephens, B. B., Baier, B. C., Chen, H., Choi, Y., Deutscher, N. M., DiGangi, J. P., Gross, J., Herkommer, B., Jeseck, P., Laemmel, T., Lan, X., McGee, E., McKain, K., Miller, J., Morino, I., Notholt, J., Ohyama, H., Pollard, D. F., Rettinger, M., Riris, H., Rousogenous, C., Sha, M. K., Shiomi, K., Strong, K., Sussmann, R., Té, Y., Velazco, V. A., Wofsy, S. C., Zhou, M., and Wennberg, P. O.: The Total Carbon Column Observing Network's GGG2020 data version, Earth Syst. Sci. Data, 16, 2197–2260, https://doi.org/10.5194/essd-16-2197-2024, 2024.
National Research Council, 2004: National Research Council, Climate Data Records from Environmental Satellites: Interim Report 2004, 150 pp., ISBN: 978-0-309-09168-8, DOI: {+}https://doi.org/10.17226/10944+, 2004.
Parkinson et al., 2006: Parkinson, C., A. Ward and M. King, Earth Science Reference Handbook, NASA, Washington DC, 2006.
PQAR MTGHG, 2024: Crevoisier, C., Meilhac, N., C3S Greenhouse Gas (GHG: MTCO2 v10.1 & MTCH4 v10.2): Product Quality Assessment Report (PQAR)
PUGS MTGHG, 2024: Crevoisier, C., Meilhac, N., C3S Greenhouse Gas (GHG: MTCO2 v10.1 & MTCH4 v10.2): Product User Guide and Specification (PUGS)
PUGS XGHG, 2024: Reuter , M., Fuentes Andrade, B., Buchwitz, M., C3S Greenhouse Gas (GHG: CO2 & CH4) v4.6: Product User Guide and Specification (PUGS), C3S project C3S2_313a_DLR, 2024. Link: https://confluence.ecmwf.int/x/HS-OIQ
Reuter et al., 2011: M. Reuter, H. Bovensmann, M. Buchwitz, J. P. Burrows, B. J. Connor, N. M. Deutscher, D. W. T. Griffith, J. Heymann, G. Keppel-Aleks, J. Messerschmidt, J. Notholt, C. Petri, J. Robinson, O. Schneising, V. Sherlock, V. Velazco, T. Warneke, P. O. Wennberg, D. Wunch: Retrieval of atmospheric CO2 with enhanced accuracy and precision from SCIAMACHY: Validation with FTS measurements and comparison with model results. Journal of Geophysical Research - Atmospheres, 116, D04301, doi: 10.1029/2010JD015047, 2011.
Reuter et al., 2020: M. Reuter, M. Buchwitz, O. Schneising, S. Noël, H. Bovensmann, J.P. Burrows, H. Boesch, A. Di Noia, J. Anand, R.J. Parker, P. Somkuti, L. Wu, O.P. Hasekamp, I. Aben, A. Kuze, H. Suto, K. Shiomi, Y. Yoshida, I. Morino, D. Crisp, C.W. O'Dell, J. Notholt, C. Petri, T. Warneke, V.A. Velazco, N.M. Deutscher, D.W.T. Griffith, R. Kivi, D.F. Pollard, F. Hase, R. Sussmann, Y.V. Té, K. Strong, S. Roche, M.K. Sha, M. De Mazière, D.G. Feist, L.T. Iraci, C.M. Roehl, C. Retscher, and D. Schepers: Ensemble-based satellite-derived carbon dioxide and methane column-averaged dry-air mole fraction data sets (2003-2018) for carbon and climate applications, Atmos. Meas. Tech., https://www.atmos-meas-tech.net/13/789/2020, 2020.
Rodgers, 2000: Rodgers, C. D.: Inverse Methods for Atmospheric Sounding: Theory and Practice, World Scientific Publishing, Singapore, 2000.
TR-GAD GHG, 2024: Buchwitz, M., et al., Target Requirement and Gap Analysis Document, Copernicus Climate Change Service (C3S) project on satellite-derived Essential Climate Variable (ECV) Greenhouse Gases (CO2 and CH4) data products (project C3S2_312a_Lot2), Version 5.6, 22-Mar-2024, pp. 87, 2024. Latest version: http://wdc.dlr.de/C3S_312b_Lot2/Documentation/GHG/C3S2_312a_Lot2_TRD-GAD_GHG_latest.pdf
Werscheck, 2015: Werscheck, M., EUMETSAT Satellite Application Facility on Climate Monitoring, C3S Climate Data Store workshop, Reading, UK, 3-6 March 2015. {+}http://www.ecmwf.int/sites/default/files/elibrary/2015/13546-existing-solutions-eumetsat-satellite-application-facility-climate-monitoring.pdf+
Wunch et al. 2010: Wunch, D., Toon, G. C., Wennberg, P. O., Wofsy, S. C., Stephens, B. B., Fischer, M. L., Uchino, O., Abshire, J. B., Bernath, P., Biraud, S. C., Blavier, J.-F. L., Boone, C., Bowman, K. P., Browell, E. V., Campos, T., Connor, B. J., Daube, B. C., Deutscher, N. M., Diao, M., Elkins, J. W., Gerbig, C., Gottlieb, E., Griffith, D. W. T., Hurst, D. F., Jiménez, R., Keppel-Aleks, G., Kort, E. A., Macatangay, R., Machida, T., Matsueda, H., Moore, F., Morino, I., Park, S., Robinson, J., Roehl, C. M., Sawa, Y., Sherlock, V., Sweeney, C., Tanaka, T., and Zondlo, M. A.: Calibration of the Total Carbon Column Observing Network using aircraft profile data, Atmospheric Measurement Techniques, 3, 1351–1362, doi:10.5194/amt-3-1351-2010, URL http://www.atmos-meas-tech.net/3/1351/2010/, 2010.
Wunch et al., 2011: Wunch, D., Toon, G. C., Blavier, J.-F. L., Washenfelder, R. A., Notholt, J., Connor, B. J., Griffith, D. W. T., Sherlock, V., and Wennberg, P. O.: The Total Carbon Column Observing Network (TCCON), Phil. Trans. R. Soc. A, 369, 2087–2112, doi:10.1098/rsta.2010.0240, 2011.
Wunch et al. 2015: Wunch, D., Toon, G.C., Sherlock, V., Deutscher, N.M., Liu, X., Feist, D.G., Wennberg, P.O., The Total Carbon Column Observing Network's GGG2014 Data Version. Carbon Dioxide Information Analysis Center, Oak Ridge National Laboratory, Oak Ridge, Tennessee, USA (available at: doi:10.14291/tccon.ggg2014.documentation.R0/1221662), 2015.
