Last modified on Jul 11, 2025 10:25

Contributors: Sebastian B. Simonsen and Natalia Havelund (Technical University of Denmark), Thomas Slater and Athul Kaitheri (Northumbria University)

Issued by: Technical University of Denmark / Sebastian B. Simonsen, Northumbria University / Thomas Slater

Date:19 Dec 2024 

Ref: C3S2_313d_ENVEO.WP2-DDP-SEC-GIS-AIS-01_202412_PQAR_v1.3

Official reference number service contract: 2024/C3S2_313d_ENVEO/SC1

History of modifications

List of datasets covered by this document

Acronyms

General definitions

Backscatter: The portion of the outgoing radar signal that the target redirects directly back toward the radar antenna.

Baseline: A combination of processor versions, auxiliary data, and other needed enablers that allows the generation of a coherent set of Earth observation products. 

Bias: The tendency of an instrument to preferentially make measurements over a certain type of surface.

Climate Data Record (CDR): A time series of measurements of sufficient length, consistency, and continuity to determine climate variability and change.

Crossover analysis: A method for deriving elevation change at locations where the orbits of a single or multiple satellites cross.

Cross-calibration: A method that merges datasets from multiple satellites into one consistent dataset. 

Key performance indicator: A quantifiable measure used to evaluate the success of a product in meeting its performance objectives.

Laser altimeter: An instrument mounted on an aircraft or spacecraft that measures altitude from the ground surface below by timing how long it takes a pulse of laser light to travel to ground, reflect, and return to the craft.

Outlier-resistant: In mathematical model fitting, outlier-resistant (or 'robust') methods exclude spurious measurements from the dataset during the modelling process.

Radar altimeter: An instrument mounted on an aircraft or spacecraft that measures altitude from the ground surface below by timing how long it takes a pulse of radio waves to travel to the ground, reflect, and return to the craft.

Stability: An estimate of the consistency of the measurements over time.

Surface Elevation Change (SEC): The surface elevation of a point on an ice sheet is the height of the ice sheet surface above a reference geoid (a hypothetical solid figure whose surface corresponds to mean sea level and its imagined extension under land areas). An increase in surface elevation over time at a given location indicates a gain of ice or snow at that location, and conversely decrease indicates a loss. The surface elevation change product provides the rate of change given at monthly intervals at each location on a grid covering the ice sheet. The definition of the grid projection includes the geoid used. Given the rates of change, absolute change can be calculated for any period of time. 

Tracking: Retrieving the radar echo from a given radar pulse.

Uncertainty: An estimate of the error in a measurement, due to limitations in the measuring instrument or statistical fluctuations in the quantity being measured.

Validation: Comparison between two independent datasets to test their agreement.

Executive summary

This document is the Product Quality Assessment Report (PQAR) for the versions 5.0 (Antarctica) and 6.0 (Greenland) of the Surface Elevation Change (SEC) products made as part of the Copernicus Ice Sheets and Ice Shelves service. The products contain geographically gridded time series of the rate of change of ice sheet surface elevation in Greenland and Antarctica, from 1992 to the present with a 3 month lag. In this document, we provide the validation results from the Climate Data Record (CDR) version 5.0 for Antarctica and version 6.0 for Greenland the surface elevation change datasets produced by the Copernicus Ice Sheets and Ice Shelves service. For version 5.0, we validated only against data from the National Aeronautics and Space Administration (NASA)'s Operation IceBridge and ICESat-2 mission, and we repeat that validation here and compare the results from the two versions. This exercise makes sure that the Antarctica and Greenland SEC products generated meet the necessary quality requirements. Version 5.0 Antarctica SEC and version 6.0 Greenland SEC meet the quality targets set by the Global Climate Observing System (GCOS) and the Copernicus Climate Change Service (C3S) project in terms of stability and accuracy. 

In section 1 we briefly describe the external datasets used in the validation, list the target requirements we aim to fulfill, and describe the CDR parameters that relate to those targets.

In section 2 we give the validation results and discuss differences seen between the product versions. 

In section 3 we describe the application-specific requirements for the Antarctic and Greenland SEC product.

In section 4 we summarise how well the product achieves the target requirements. 

The products are hosted on the Copernicus Climate Data Store.

1. Product validation methodology

1.1. Validation data

Before turning to the methodology of model validation, the two external validation datasets are presented.

1.1.1. Operation IceBridge Airborne Topographic Mapper

The Airborne Topographic Mapper (ATM) is a scanning laser altimeter flown on board aircraft by NASA Operation IceBridge (OIB) (Studinger 2014). The ATM instrument has been flown in Greenland since 1993, and within the OIB data package are estimates of surface change from all repeat measurements of surface elevation done by OIB. In Antarctica, it has been flown since 2002, but at the time of writing, SEC data is only available up to 2019, i.e. it is unchanged since the version 3.0 validation. For further details see the related Product Quality Assurance Document where both the IceBridge dataset and the validation methodology are described in detail.

1.1.2. ICESat-2 ATL15

The ICESat-2 ATL15² (Smith 2021a) data product is a gridded surface height change product over land ice. This dataset is derived from the Advanced Topographic Laser Altimeter System (ATLAS) aboard NASA's ICESat-2 satellite. ATL15 is derived from the higher-order data product of collocated height measurements provided by ATL11 (Smith 2021b). In the following validations, the data subset "dhdt_lag8"³  is used, which provides biennial height-change-rate estimates in the period late-2019 to mid-2020.

1.2. Surface elevation change, Antarctica

As in the previous versions, the CDR version 5.0 is validated against data provided by the Operation IceBridge ATM first. For the validation of surface elevation change (SEC), we compare OIB ATM L4 Surface Elevation Rate of Change V001 product, and to contemporaneous C3S SEC measurements. This is not a direct comparison to the product dataset, as the ATM overflights are highly irregular. Instead, we use the underlying elevation change time-series and analysis methods from which the product dataset was derived to produce results comparable to the ATM. We averaged the rates of elevation change computed from pairs of ATM overflights in 25 km cells to match the spatial resolution of the satellite data. Similarly, we averaged the rates of elevation change, derived by linear least-squares fitting from the surface elevation change data, comparable to the ATM pairs in length of time and grid cell location. The comparison was restricted to where the root mean square (rms) of the ATM results and the standard deviation of the satellite results was less than or equal to 0.4 m/year, to only use results of good quality (taken from the 3-sigma clipped mean rms of the ATM dataset). It was also restricted to where the time period between ATM overflights was at least 2 years, to allow for the inclusion of at least two seasonal elevation change cycles. A flowchart of the method for producing comparable datapoints is given in Figure 1.

Figure 1. Flowchart for producing a pair of datapoints, one from the ATM and one from the SEC product, which can be directly compared. The full validation dataset uses all comparable pairs from each grid cell and ATM period.

We summarise the agreement between these datasets in Table 2. Because OIB stopped operating in 2018, we also validate C3S SEC data against the ICESat-2 ATL15 dataset which gives 20 km grids of surface elevation change rates calculated over a 2-year window. The methodology is similar to that of the IceBridge validation, using the underlying CDR data to match the ATL15 time and location ranges as far as possible, but involves an extra step as the ATL15 grid has to be reprojected to match the CDR grid.

As well as validation against external data, to comply with user requirements gathered by the Global Climate Observing System (GCOS) for its Implementation Plan (2022), the product should achieve two statistical targets in terms of stability and accuracy. Moreover, the Copernicus Climate Change Service (C3S) project has identified two additional key performance indicators, as shown in Table 1.

Table 1: Antarctic surface elevation change product targets. 

The stability is taken as one standard deviation of the linear model fit to the surface elevation change time series used in deriving the surface elevation change rate. 

The accuracy is the total error budget for the surface elevation change rate, which has three components - the input uncertainty, the uncertainty due to cross-calibration (if applicable), and the stability, as summarized below. 

The input uncertainty is the standard deviation of the surface elevation change values within the aggregation area, i.e. within a single pixel or within a basin.

The uncertainty due to cross-calibration is the standard deviation of the cross-calibration adjustment estimate. The cross-calibration method uses multiple regression, using the REGRESS function in the IDL v8 software package. It calculates the adjustments, single values added to each data point per mission, necessary to make a consistent multi-mission time-series. The function provides uncertainty estimates for each adjustment found from the regression, given input uncertainties as described above. 

The three components are summed in quadrature to produce the total error. This method is applicable at both basin and pixel level.

The product only supplies pixel-level data. It can be used to make basin-level datasets, but the method employed should be selected by the user to best match their needs. The basin-level datasets used to check against the targets are made with the methodology used in Shepherd et al (2019), where the data gaps in any basin at a given time are filled by an ice-velocity-guided value derived from the rest of the basin, using the Berkeley – Ice Sheet Initiative for Climate Extremes (BISICLES) ice velocity model, see Cornford et al (2013).

1.3. Surface elevation change, Greenland

The Greenland surface elevation changes are validated against data provided by OIB ATM first. This higher-level product (level-4) has been downloaded from National Snow and Ice Data Center (NSIDC⁴) For the validation of the surface elevation change (SEC), we use the monthly dh estimates to derive the cumulative surface elevation change (dh/dt), thereby reproducing the temporal span in-between repeat observations by the OIB ATM. As the OIB data are usually obtained once a year (in spring), we summarize the mean and standard deviation of the difference in SEC between OIB and C3S SEC here. This mean and standard deviation of the estimated difference is ascribed to the year of the first ATM observation.

Secondly, a validation effort is conducted against ICESat-2 ATL15 data. The ATL15 product provided 2-year averaged surface elevation changes at 20 km grid posting. In line with the above presented ATM validation, we derive cumulative surface elevation change (DH/dt) from the C3S SEC at the same time span as the 2-year averaging in the ATL15 product. The ALT15 data is further regridded to the 25 km grid posting of the C3S CDR SEC product.  

As well as validation against external data, to comply with user requirements gathered by GCOS, the product should achieve two statistical targets. Moreover, the C3S project has identified two Key Performance Indicators (KPIs), as shown in Table 2. The Greenland surface elevation change uses the KPIs outlined in Table 2, except for the surface coverage which is obsolete due to the more optimal location of the Greenland ice sheet in relation to the satellite orbits and differences in postprocessing interpolation. More details on the target requirements for the SEC product are discussed in the Target Requirements and Gap Analysis Document (TRGAD)

Table 2: Greenland surface elevation change product targets. 

Here, the stability is taken as one standard deviation of the linear model fit to the surface elevation change time series used in deriving the surface elevation change rate whereas the accuracy is the total error budget for the surface elevation change rate. This, along with a more detailed discussion of the uncertainty processing is discussed in the Algorithm Theoretical Basis Document.

2.  Validation results

2.1. Surface elevation change, Antarctica 

Table 3 shows the results of the comparison between the IceBridge data and the CDR version 5.0 and version 4.0. All IceBridge Antarctic data available from the portal, as of 1st October 2022, was considered. However, since there have been no Antarctic updates on the portal in the last year, the same dataset was used for both the version 5.0 and version 4.0 validations. For operational reasons, IceBridge flights were concentrated on the West Antarctic Ice Sheet, Filchner-Ronne Ice Shelf, and surrounding regions. Figure 2 summarises validation of CDR versions 5.0 and 4.0 against Operation IceBridge with A) the locations for which comparison was possible, B) a scattergram of the values compared, and C) a histogram of their differences. 

The outlier-resistant cell-averaged mean difference (i.e., excluding outliers of more than 3 sigma) is well within the GCOS surface elevation change target accuracy. The histogram shows the peak difference is close to zero and the points of the scattergram lie along or close to the line of equality, as expected. 

Table 3: Antarctic SEC validation against Operation IceBridge 

Figure 2. A) Locations of Operation IceBridge datapoints used in Antarctic validation. B)  Scattergram of SEC in the same grid cell from both Operation IceBridge and CDRs versions 5.0 and 4.0. C) Histogram of SEC differences per grid cell between Operation IceBridge and CDRs version 5.0 and 4.0.

The CDR version 5.0 dataset is mainly a continuation of version 4.0, including new data from January 2023 onwards from CryoSat-2, Sentinel-3A and Sentinel-3B. This also includes using new the Baseline E product for CryoSat-2 and land ice thematic data products for Sentinel-3A and Sentinel-3B. For CryoSat-2, Baseline E improves the land ice waveform retracking in both altimeter operating modes and resolves an issue with the computation of backscatter power in low resolution mode (LRM) which caused a small drift in expected values in the previous baseline. For Sentinel-3A and Sentinel-3B, the thematic land ice product provides significantly improved data coverage. This allows more data retrieval in the ice sheet margins, where the majority of changes in Antarctica occur. See the Algorithm Theoretical Basis Document for details.

Figure 3 shows the statistical analysis of the version 5.0 dataset. The histograms of dataset accuracy show both the component and total figures. The three components are: 

• epoch uncertainty – the uncertainty at each measurement epoch, which combines all sources of error from the input data 

• cross-calibration uncertainty – the uncertainty in the adjustment applied to the time series from each mission to produce a final, continuous multi-mission time series 

• model uncertainty – the uncertainty in the fitting of the linear model used to derive the SEC, which is also the measure of the data stability

Figure 3 shows that the dominant uncertainty comes from the altimetry measurements (the epoch uncertainty) – its distribution has the least percentage of instances within the GCOS target, and its peak is at the highest value. At the pixel level the observations are closely clustered, but at basin level they incorporate both (a) interpolation to fill data gaps and (b) a wide range of terrain, and thus the overall uncertainty is higher. 

Coverage depends on altimetry mission. European Remote-sensing Satellite (ERS) 1, ERS2, Envisat, Sentinel-3A and Sentinel-3B are unable to observe within 8.5° of the South Pole, so 20% of the Antarctic Ice Sheet cannot be covered by these missions. However, they all follow repeating orbits, which allows for regular crossover analysis across the observed regions. CryoSat-2 can observe within 2° of the pole, excluding only 1% of the Antarctic Ice Sheet, but has a drifting orbit on a very long repeat cycle. This makes crossover analysis more challenging, especially in coastal regions where surface slope is high and altimetric measurements are not always possible. To better represent the CryoSat-2 contribution, coverage results are aggregated yearly. 

In practice, even when CryoSat-2 near-polar data is included, marginal crossover performance is relatively poor, and the higher target is not achieved. For example, very little data is retrieved by any mission from the rugged terrain of the Antarctic Peninsula. The coverage dip in 2011 comes from declining amounts of Envisat data combined with the initiation of the CryoSat-2 coverage. Most surface elevation change rate values come from data from a mix of missions, but there is a coverage dip centered on 2014, when (briefly) only CryoSat-2 data was available within the 5-year SEC period spans. However, the complementary orbital configuration of the Sentinels, when used together, improves the coverage of the later periods. 

It should be noted that the current release of Sentinel-3A and B data used in version 5.0 has overcome the problems documented in version 4.0 and provides increased coverage.  

Figure 3: Statistical summary from Antarctic surface elevation change product.  A) Pixel-level accuracy, B) basin-level accuracy and C) percentage coverage of the Antarctic ice sheets and ice shelves region. The accuracy statistics are shown by total and component, and the percentage of instances within the GCOS target is given.

The validation against ICESat-2 was performed for the four time periods listed in Table 4, and maps of the averaged CDR and ICESat-2 data and their difference are shown in Figure 4. There is a known issue⁵ with the ATL15 dataset over the Antarctic, where the time-difference between repeat tracks leads to them sampling dynamic ice events differently, resulting in some 'streakiness' across the ice sheets. The validation performed here is thus illustrative only. 

Table 4: Antarctic SEC validation against ICESat-2

Figure 4: Maps of A) Antarctic SEC and B) ICESat-2 SEC comparison data and C) their differences.

2.2. Surface elevation change, Greenland

Figure 5 illustrates the results of the OIB ATM intercomparison, incorporating all available Greenland OIB elevation change data from the NSIDC as of November 2023. The right panel of the figure displays the geolocated differences between OIB and C3S observations, presenting a point-to-point intercomparison and the distribution of these differences.

Validation against ICESat-2 ATL15 was conducted for the specified time periods detailed in Table 5 and Figure 6. The analysis revealed an anticipated bias between the lidar observation (ATL15) and the radar, attributed to differences in scattering horizons, consistent with findings in existing literature (Simonsen and Sørensen 2017).

In the updated Version 6, it is a complete reprocessing of the CDR with the new ArcticDEM version and the statistics align well with the previous version. 

Figure 5:  (Panel a) The point-to-point correlations between the C3S-surface elevation change and the OIB surface elevation estimate. (Panel b) The histogram of the point-to-point differences between C3S and OIB surface elevation change. The left panels also show the old version of the CDR (v5) for reference. (Panel c) The spatial distribution of the point-to-point inter-comparison with the OIB ATM surface elevation estimates.

Table 5: Results of ICESat-2 ATL15 validation for the Greenland version 6.0 product. The results for the previous version (version 5.0) are also shown for comparison.

Figure 6: Validation against ICESat-2 ALT15 average elevation change from January 2019 to December 2020 In the first panel to the left is the C3S data product,

in the middle panel is the IceSat-2 data product, and to the right is the difference between the two shown. 

Additionally, Figure 7 shows the statistics from the internal-fitting accuracy, showing that the two data products align closely with minimal differences.

Figure 7: The stability in the linear model fit to the surface elevation change time series used in deriving the surface elevation change rate. For reference, the C3S version 5.0 estimate is also included but lies under the red line as the two data products align closely with minimal differences. As seen, the new CDR (version 6.0) fulfills the GCOS requirements (accuracy in the left panel) and the fitting stability (right panel)

3. Climate Change Assessment (to be implemented progressively)

This section will be updated with the outputs of the Climate Intelligence activities as they become available.

4. Application(s) specific assessments

4.1. Surface elevation change, Antarctica 

The Antarctic Surface Elevation Change (SEC) product has been assessed by multiple stakeholders for scientific research, environmental monitoring, and policy-making. The product demonstrates consistency both spatially and temporally and is compared with independent observations from Operation IceBridge and ICESat-2 before release. It is found to be reliable and can be integrated with other ECVs for climate studies. Some "stripiness" can occur in the data. 

4.2. Surface elevation change, Greenland

The Greenland Surface Elevation Change (SEC) product has been assessed for its applicability in monitoring ice sheet dynamics and supporting climate-related studies. It demonstrates consistent spatial patterns of elevation change when compared with independent datasets such as ICESat-2 ATL15 and Operation IceBridge, supporting its reliability for use in mass balance assessments and integration with other Essential Climate Variables (ECVs).

5. Compliance with user requirements

5.1. Surface elevation change, Antarctica 

In validation, the cell-averaged mean difference (excluding outliers beyond 3 standard deviations) between the independent and C3S datasets was -0.054 ± 0.635 m/yr. This is within the GCOS target accuracy of 0.1 m/yr. 

Table 6 provides an overview of the product compliance with targets and KPIs. A discussion of the reasons for partial non-compliance is given in Section 2.1. 

Table 6: Antarctic SEC compliance with targets 

5.2. Surface elevation change, Greenland

The uncertainty given in the product is derived from the combination of the epoch (derived from the supplied input data) and the model (derived from the plane-fitting) uncertainties. All the analysis and evaluation process is shown in the three figures presented in section 2.2 (Figure 6 to Figure 8).

The number of points with an accuracy within the GCOS requirements is 99.9%. This estimate is purely an internal stability indication and not a real error estimate, as multiple factors are not included in this estimate. Factors such as changes in the penetration depth of the radar, slope-induced relocation errors, and the fact that the radar only observes changes at the highest point within its footprint, may bias the elevation change estimate.

Therefore, the real error estimate and the number needed to fulfill the user-requirements needs are to be found by applying independent validation of the surface elevation change product. Here, the independent validation is provided by comparison to NASA's OIB airborne laser altimetry campaigns and ICESat-2 ATL15. Figure 6 shows the result of the inter-comparison between the OIB and C3S-Greenland SEC, with a mean bias of all observations of -1.59 cm per year. Therefore, the current processing version fulfills the GCOS requirement (given in Table 2). This is further confirmed with the new ICESat-2 data all showing similar biases.

Table 7 provides an overview of the product compliance with targets and KPIs.

Table 7: Greenland SEC compliance with targets 

References

Cornford et al. (2013). Adaptive mesh, finite volume modelling of marine ice sheets, Journal of Computational Physics, 232(1):529-549
ESA Mission Management, Copernicus S3 Product Notice – Altimetry, July 2022, available at https://sentinel.esa.int/documents/247904/2753172/Sentinel-3-Product-Notice-LAND-L1-L2-NRT-STC-NTC.pdf (URL resource last viewed 9^th December 2024)

Global Climate Observing System (2022). The 2022 GCOS Implementation Plan (GCOS-244), available at https://gcos.wmo.int/index.php/en/publications/gcos-implementation-plan2016 (URL resource last viewed 9^th December 2024)

Shepherd et al. (2019). Trends in Antarctic ice sheet elevation and mass. Geophysical Research Letters. https://doi.org/10.1029/2019GL082182  https://doi.org/10.1029/2019GL082182 

Simonsen, Sebastian B., and Louise Sandberg Sørensen. (2017). Implications of Changing Scattering Properties on Greenland Ice Sheet Volume Change from Cryosat-2 Altimetry. Remote Sensing of Environment. https://doi.org/10.1016/j.rse.2016.12.012 (URL resource last viewed 9^th December 2024)

Smith, B., B. P. Jelley, S. Dickinson, T. Sutterley, T. A. Neumann, K. Harbeck. (2021a). ATLAS/ICESat-2 L3B Gridded Antarctic and Arctic Land Ice Height Change, Version 1. Boulder, Colorado USA. NASA National Snow and Ice Data Center Distributed Active Archive Center. https://doi.org/10.5067/ATLAS/ATL15.001 (URL resource last viewed 9^th December 2024)

Smith, B. (2021b). ICESat-2 Algorithm Theoretical Basis Document for Land Ice DEM and Land Ice Height Change Release 001 Algorithm Theoretical Basis Document (ATBD) for Land-ice DEM (ATL14) and Land-ice height change (ATL15). https://icesat-2.gsfc.nasa.gov/sites/default/files/page_files/ICESat2_ATL14_ATL15_ATBD_r001.pdf (URL resource last viewed 9^th December 2024)

Studinger, M. (2014). IceBridge ATM L4 Surface Elevation Rate of Change, Version 299 1, Antarctica subset. N. S. a. I. D. C. D. A. A. Center. Boulder, Colorado, USA. DOI: 10.5067/BCW6CI3TXOCY

Related articles

• Page:
  SEC Antarctica Version 5.0 and Greenland Version 6.0: Product User Guide and Specification (PUGS)
• Page:
  SEC Antarctica Version 5.0 and Greenland Version 6.0: Product Quality Assessment Report (PQAR)
• Page:
  SEC Antarctica Version 5.0 and Greenland Version 6.0: Algorithm Theoretical Basis Document (ATBD)
• Page:
  Ice Sheets v2024: Target Requirements and Gap Analysis Document (TRGAD)
• Page:
  SEC Version 4.0: Product Quality Assessment Report (PQAR)