Last modified on Dec 01, 2023 17:38

Contributors: L. Gilbert (University of Leeds), S. B. Simonsen (Technical University of Denmark)

Issued by: University of Leeds / Lin Gilbert

Date: 17/08/2023

Ref: C3S2_312a_Lot4.WP2-FDDP-IS-v1_202212_SEC_PQAR-v4_i1.1

Official reference number service contract: 2021/C3S_312a_Lot4_EODC/SC1

History of modifications

List of datasets covered by this document

Related documents

Acronyms

General definitions

Backscatter: The portion of the outgoing radar signal that the target redirects directly back toward the radar antenna.

Baseline: A combination of processor versions, auxiliary data, and other needed enablers that allows the generation of a coherent set of Earth observation products. 

Bias: The tendency of an instrument to preferentially make measurements over a certain type of surface.

Climate Data Record (CDR): A time series of measurements of sufficient length, consistency, and continuity to determine climate variability and change.

Crossover analysis: A method for deriving elevation change at locations where the orbits of a single or multiple satellites cross.

Cross-calibration: A method that merges datasets from multiple satellites into one consistent dataset. 

Key performance indicator: A quantifiable measure used to evaluate the success of a product in meeting its performance objectives.

Laser altimeter: An instrument mounted on an aircraft or spacecraft that measures altitude from the ground surface below by timing how long it takes a pulse of laser light to travel to ground, reflect, and return to the craft.

Outlier-resistant: In mathematical model fitting, outlier-resistant (or 'robust') methods exclude spurious measurements from the dataset during the modelling process.

Radar altimeter: An instrument mounted on an aircraft or spacecraft that measures altitude from the ground surface below by timing how long it takes a pulse of radio waves to travel to ground, reflect, and return to the craft.

Stability: An estimate of the consistency of the measurements over time.

Surface Elevation Change (SEC): The surface elevation of a point on an ice sheet is the height of the ice sheet surface above a reference geoid (a hypothetical solid figure whose surface corresponds to mean sea level and its imagined extension under land areas). An increase in surface elevation over time at a given location indicates a gain of ice or snow at that location, and conversely decrease indicates a loss. The surface elevation change product provides the rate of change given at monthly intervals at each location on a grid covering the ice sheet. The definition of the grid projection includes the geoid used. Given the rates of change, absolute change can be calculated for any period of time. 

Tracking: Retrieving the radar echo from a given radar pulse.

Uncertainty: An estimate of the error in a measurement, due to limitations in the measuring instrument or statistical fluctuations in the quantity being measured.

Validation: Comparison between two independent datasets to test their agreement.

Scope of the document

This document is the Product Quality Assessment Report for the Copernicus Ice Sheets and Ice Shelves Service, surface elevation change (SEC) essential climate variable (ECV). It presents the results of the quality assessment for the provided Antarctic and Greenland datasets and a discussion of how well the Global Climate Observing System (GCOS) and user requirements have been met.

The service addresses three ECVs by providing four separate products:

• Ice velocity is given for Greenland in product WP2-FDDP-IV-CDR
• Gravimetric mass balance is given for Greenland and Antarctica in product WP2-FDDP-GMB-CDR
• Surface elevation change is given for:
  • Antarctica in product WP2-FDDP-SEC-CDR-AntIS
  • Greenland in product WP2-FDDP-SEC-CDR-GrIS

The products are hosted on the Copernicus Climate Data Store at https://cds.climate.copernicus.eu/cdsapp#!/dataset/satellite-ice-sheet-elevation-change?tab=overview¹

Executive summary

In this document, we provide the validation results from the CDR v4.0 for the surface elevation change datasets produced by the Copernicus Ice Sheets and Ice Shelves service. For v3.0, we validated only against data from National Aeronautics and Space Administration (NASA)'s Operation IceBridge, and we repeat that validation here, and compare the results from the two versions. For v4.0 there is an update as we include an experimental validation against data from NASA's ICESat-2 mission, which recently released a gridded SEC dataset.

In section 1 we briefly describe the external datasets used in the validation, list the target requirements we aim to fulfil, and describe the CDR parameters that relate to those targets.

In section 2 we give the validation results and discuss differences seen between the v4.0 and v3.0 products. In both the Greenland and Antarctic products, we see a small improvement in agreement with the validating datasets. This is expected, as the v4.0 product, although upgraded, is meant to provide continuity with v3.0.

Section 3 is not applicable, as there are no application-specific requirements for these products.

In section 4 we summarise how well the products achieve the target requirements. Each dataset fulfills the targets for accuracy and stability. The Antarctic product's key performance indicator (KPI) for coverage is not achieved over its full period.

1. Product validation methodology

1.1. Validation data

Before turning to the methodology of model validation, the two external validation datasets are presented.

1.1.1. Operation IceBridge Airborne Topographic Mapper

The Airborne Topographic Mapper (ATM) is a scanning laser altimeter flown on board aircraft by NASA Operation IceBridge (OIB) (Studinger 2014). The ATM instrument has been flown in Greenland since 1993, and within the OIB data package are estimates of surface change from all repeat measurements of surface elevation done by OIB. In Antarctica it has flown since 2002, but at time of writing, SEC data is only available up to 2016, i.e. it is unchanged since the v3.0 validation. For further details see the related Product Quality Assurance Document [RD2] where both the IceBridge dataset and the validation methodology are described in detail.

1.1.2. ICESat-2 ATL15

As a part of NASA's ICESat-2 laser altimeter data releases a gridded surface height change product over land ice, named ATL15², has become available in the last year (Smith 2021a). ATL15 is derived from the higher-order data product of collocated height measurements provided by ATL11 (Smith 2021b). In the following validations, the data subset "dhdt_lag8"  is used, which provides biennial height-change-rate estimates in the period late-2019 to mid-2020.

1.2. Surface elevation change, Antarctica

As for the previous versions,the CDR v4.0 is validated against data provided by the Operation IceBridge ATM.A comparison is made between the OIB ATM L4 Surface Elevation Rate of Change V001 product, and a dataset matching its measurements in location and time, calculated from the Antarctic surface elevation change product's underlying data.

Furthermore, for the v4.0 there is an update in the process, the product is also validated against the ICESat-2 ATL15 dataset, specifically the 20km resolution gridded dataset, which gives grids of surface elevation change rates calculated in a 2-year window. The methodology is similar to that of the IceBridge validation, using the underlying CDR data to match the ATL15 time and location ranges as far as possible, but involves an extra step as the ATL15 grid has to be reprojected to match the CDR grid.

As well as validation against external data, as discussed in the TRGAD [RD3], to comply with user requirements gathered by the Global Climate Observing System (GCOS) for its Implementation Plan (2016), the product should achieve two statistical targets. Moreover, the Copernicus Climate Change Service (C3S) project has identified two additional key performance indicators, as shown in Table 1.1.

Table 1.1: Antarctic surface elevation change product targets. These targets are also valid for the Greenland surface elevation change product.

The stability is taken as one standard deviation of the linear model fit to the surface elevation change time series used in deriving the surface elevation change rate.

The accuracy is the total error budget for the surface elevation change rate, which has three components - the input uncertainty, the uncertainty due to cross-calibration (if applicable), and the stability, as summarized below.

The input uncertainty is the standard deviation of the surface elevation change values within the aggregation area, i.e. within a single pixel or within a basin.

The uncertainty due to cross-calibration is the standard deviation of the cross-calibration adjustment estimate. The cross-calibration method uses multiple regression, using the REGRESS function in the IDL v8 software package. It calculates the adjustments, single values added to each datapoint per mission, necessary to make a consistent multi-mission timeseries. The function provides uncertainty estimates for each adjustment found from the regression, given input uncertainties as described above.

The three components are summed in quadrature to produce the total error. This method is applicable at both basin and pixel level.

The product only supplies pixel-level data. It can be used to make basin-level datasets, but the method employed should be selected by the user to best match their needs. The basin-level datasets used to check against the targets are made with the methodology used in Shepherd et al (2019), where the data gaps in any basin at a given time are filled by an ice-velocity-guided value derived from the rest of the basin, using the Berkeley – Ice Sheet Initiative for Climate Extremes (BISICLES) ice velocity model, see Cornford et al (2013). This, along with a more detailed discussion of the uncertainty processing, is discussed in the Algorithm Theoretical Basis Document [RD1].

1.3. Surface elevation change, Greenland

In line with the Antarctica surface elevation changes, the Greenland counterpart is validated against data provided by OIB ATM first. This higher-level product (level-4) has been downloaded from National Snow and Ice Data Center (NSIDC⁴) For the validation of the surface elevation change (SEC), we use the monthly dh estimates to derive the cumulative surface elevation change (dh/dt), thereby reproducing the temporal span in-between repeat observations by the OIB ATM. As the OIB data are usually obtained once a year (in spring), we summarize the mean and standard deviation of the difference in SEC between OIB and C3S SEC here. This mean and standard deviation of the estimated difference is ascribed to the year of the first ATM observation.

Secondly, a validation effort is conducted against ICESat-2 ATL15 data. Here, the ATL15 product provided 2-year averaged surface elevation changes at 20 km grid posting. In line with the above presented ATM validation, we derive cumulative surface elevation change (dh/dt) from the C3S SEC at the same time span as the 2-year averaging in the ATL15 product. The ALT15 data is further regridded to the 25 km grid posting of the C3S CDR SEC product.  

As well as validation against external data, to comply with user requirements gathered by GCOS, the product should achieve two statistical targets. Moreover the C3S project has identified two KPIs, as shown in Table 1.1 for Antarctica. The Greenland surface elevation change uses the KPIs outlined in Table 1.1, with the exception of the surface coverage which is obsolete due to the more optimal location of the Greenland ice sheet in relation to the satellite obits and differences in postprocessing interpolation.

Here, the stability is taken as one standard deviation of the linear model fit to the surface elevation change time series used in deriving the surface elevation change rate whereas the accuracy is the total error budget for the surface elevation change rate. This, along with a more detailed discussion of the uncertainty processing is discussed in the Algorithm Theoretical Basis Document [RD1].

2. Validation results

2.1. Surface elevation change, Antarctica

Table 2.1 shows the results of the comparison between the IceBridge data and the CDR v4.0 and v3.0. All IceBridge Antarctic data available from the portal³, as of 1st October 2022, was considered. However, since there have been no Antarctic updates on the portal in the last year, the same dataset was used for both the v3.0 and v4.0 validations. For operational reasons, IceBridge flights were concentrated on the West Antarctic Ice Sheet, Filchner-Ronne Ice Shelf, and surrounding regions. Figure 2.1 shows the locations for which comparison was possible, Figure 2.2 a scattergram of the values compared, and Figure 2.3 a histogram of their differences.

The outlier-resistant cell-averaged mean difference (i.e., excluding outliers of more than 3 sigma) is well within the GCOS surface elevation change target accuracy. The histogram shows the peak difference is close to zero and the points of the scattergram lie along or close to the line of equality, as expected.

Table 2.1: Antarctic SEC validation against Operation Ice Bridge

Figure 2.1: Locations of Operation IceBridge datapoints used in Antarctic validation.

Figure 2.2: Scattergram of SEC in the same grid cell from both Operation IceBridge and CDRs v4.0 and v3.0.

Figure 2.3. Histogram of SEC differences per grid cell between Operation IceBridge and CDRs v4.0 and v3.0.

The CDR v4.0 dataset is mainly a continuation of v3.0, including new data from October 2022 onwards from CryoSat-2, Sentinel-3A and Sentinel-3B. The only changes with respect to the algorithm were made to the filtering of the data, using a cell-based approach rather than a general cut-off value. See the Algorithm Theoretical Basis Document [RD1] for details. This allowed more data retrieval in dynamic regions but removed a small amount of data from the interior of the ice sheet, which previously was below the general cut-off criterion, but was now deemed to be too outlying compared to the data distribution in the same cell. A lot of the validation datapoints are in dynamic areas, and the slight improvement in the v4.0 validation statistics suggests that the new filtering scheme has not affected the quality of the data.

Figure 2.4 shows the statistical analysis of the v4.0 dataset. The histograms of dataset accuracy show both the component and total figures. The three components are:

• epoch uncertainty – the uncertainty at each measurement epoch, which combines all sources of error from the input data
• cross-calibration uncertainty – the uncertainty in the adjustment applied to the time series from each mission to produce a final, continuous multi-mission time series
• model uncertainty – the uncertainty in the fitting of the linear model used to derive the SEC, which is also the measure of the data stability

Figure 2.4 shows that the dominant uncertainty comes from the altimetry measurements (the epoch uncertainty) – its distribution has the least percentage of instances within the GCOS target, and its peak is at the highest value. At the pixel level the observations are closely clustered, but at basin level they incorporate both (a) interpolation to fill data gaps and (b) a wide range of terrain, and thus the overall uncertainty is higher.

Coverage depends on altimetry mission. European Remote-sensing Satellite (ERS) 1, ERS2, Envisat, Sentinel-3A and Sentinel-3B are unable to observe within 8.5° of the South Pole, so 20% of the Antarctic Ice Sheet cannot be covered by these missions. However, they all follow repeating orbits, which allows for regular crossover analysis across the observed regions. CryoSat-2 can observe within 2° of the pole, excluding only 1% of the Antarctic Ice Sheet, but has a drifting orbit on a very long repeat cycle. This makes crossover analysis more challenging, especially in coastal regions where surface slope is high and altimetric measurements are not always possible. To better represent the CryoSat-2 contribution, coverage results are aggregated yearly.

In practice, even when CryoSat-2 near-polar data is included, marginal crossover performance is relatively poor, and the higher target is not achieved. For example, very little data is retrieved by any mission from the rugged terrain of the Antarctic Peninsula. The coverage dip in 2011 comes from declining amounts of Envisat data combined with the initiation of the CryoSat-2 coverage. Most surface elevation change rate values come from data from a mix of missions, but there is a coverage dip centered on 2014, when (briefly) only CryoSat-2 data was available within the 5-year SEC period spans. However, the complementary orbital configuration of the Sentinels, when used together, improves the coverage of the later periods.

It should be noted that the current release of Sentinel-3A and B data has known problems when the satellite track crosses from ocean to land, leading to loss of data in coastal regions. For this reason, the Sentinel-3A and B datasets have lower coverage than expected. This problem will be fixed with a new land-ice specific processor which will be used to reprocess all the mission data, but unfortunately the full reprocessed dataset was not scheduled for release before the end of 2022⁵, which was too late for its inclusion in the v4.0 product.

Figure 2.4. Statistical summary from Antarctic surface elevation change product. Top to bottom – pixel-level accuracy, basin-level accuracy and percentage coverage of the Antarctic ice sheets and ice shelves region. The accuracy statistics are shown by total and component, and the percentage of instances within the GCOS target are given.

The validation against ICESat-2 was performed for the four time periods listed in Table 2.2, and maps of the averaged CDR and ICESat-2 data and their difference are shown in Figure 2.5. There is a known issue⁶ with the ATL15 dataset over the Antarctic, where the time-difference between repeat tracks leads to them sampling dynamic ice events differently, resulting in some 'streakiness' across the ice sheets. The validation performed here is thus illustrative only.

Table 2.2: Antarctic SEC validation against ICESat-2

 
Figure 2.5: Maps of Antarctic SEC and ICESat-2 comparison data and their differences

2.2. Surface elevation change, Greenland

Figure 2.6 shows the OIB ATM intercomparison results. Here, all Greenland OIB elevation change data available from the NSIDC, as of November 2022, was considered. The figure shows the geolocated difference between the OIB and C3S observations (right panel), the point-to-point intercomparison, and the distribution of the differences. The version 4 solution shows improvements in the overall fitting stability and many of the outliers seen in version 3 have been removed. The majority of the OIB observations are located at the fast-changing outlet glaciers, which biases the statistics as the radar observations applied in the C3S elevation change estimates struggle to resolve the short time scales at which the elevation change is occurring at these outlet glaciers. This gives rise to the broader distribution towards negative values as shown in Figure 2.6b. With the version 4 updates, we observe a slightly increased bias (median difference to OIB) compared to version 3; it is now 2 cm per year, however the fitting stability is improved in version 4.

The validation against ICESat-2 ATL15 was performed for the four time periods listed in Table 2.3 and Figure 2.7. Here, we observed the expected bias between the lidar observation (ATL15) and the radar due to differences in scattering horizons, as also seen in the literature (Simonsen and Sørensen 2017).

Figure 2.6: (Panel c) The spatial distribution of the point-to-point inter-comparison with the OIB ATM surface elevation estimates. (Panel a) The point-to-point correlations between the C3S-surface elevation change and the OIB surface elevation estimate. (Panel b) The histogram of the point-to-point differences between C3S and OIB surface elevation change. The left panels also shows the old version of the CDR (v3) for refence

Table 2.3: Results of ICESat-2 ATL15 validation

Figure 2.7: Validation against ICESat-2 ALT15 average elevation change from July 2019 to July 2021.

Additionally, Figure 2.8 shows the statistics from the internal-fitting accuracy. Here we see an improvement with the update to version 4.

Figure 2.8: The stability in the linear model fit to the surface elevation change timeseries used in deriving the surface elevation change rate. For reference, the C3S v3 estimate is also included. As seen, the new CDR (v4) provides a clear improvement in relation to the GCOS requirements (accuracy in the left panel) and the fitting stability (right panel)

3. Application(s) specific assessments

3.1. Surface elevation change, Antarctica

Some "stripiness" can occur in the data, see further explanation below in Section 3.2.

3.2. Surface elevation change, Greenland

The user should note that the applied satellite data originate from a nadir-looking altimeter, hence no direct observations of SEC are available between satellite tracks. Here we rely on the ordinary kriging method to interpolate data, however, the final solution still contains some "stripiness" due to slight differences in observational periods. By applying ordinary kriging, the distance to the nearest observational point is included in the error estimate and therefore the stripiness is accounted for in the final SEC estimate.

4. Compliance with user requirements

4.1. Surface elevation change, Antarctica

In validation, the cell-averaged mean difference (excluding outliers beyond 3 standard deviations) between the independent and C3S datasets was -0.017 ± 0.624 m/yr. This is within the GCOS target accuracy of 0.1 m/yr.

Table 4.1 provides overview of the product compliance with targets and KPIs. Discussion of the reasons for partial non-compliance is given in Section 2.1.

Table 4.1: Antarctic SEC compliance with targets

4.2. Surface elevation change, Greenland

The uncertainty given in the product is derived from the combination of the epoch (derived from the supplied input data) and the model (derived from the plane-fitting) uncertainties. All the analysis and evaluation process is shown in the three figures presented in section 2.2 (Figure 2.6 to Figure 2.8).

The number of points with an accuracy within the GCOS requirements is 99.9%. This estimate is purely an internal stability indication and not a real error estimate, as multiple factors are not included in this estimate. Factors such as changes in the penetration depth of the radar, slope-induced relocation errors, and the fact that the radar only observes changes at the highest point within its footprint, may bias the elevation change estimate.

Therefore, the real error estimate and the number which needs to fulfill the user-requirements needs are to be found by applying independent validation of the surface elevation change product. Here, the independent validation is provided by comparison to NASA's OIB airborne laser altimetry campaigns and ICESat-2 ATL15. Figure 2.6 shows the result of the inter-comparison between the OIB and C3S-Greenland SEC, with a mean bias of all observations of 2.0 cm per year. Thereby, the current processing version fulfils the GCOS requirement (given in Table 1.1). This is further confirmed with the new ICESat-2 data all showing similar biases.

References

Cornford et al. (2013). Adaptive mesh, finite volume modelling of marine ice sheets, Journal of Computational Physics, 232(1):529-549
ESA Mission Management, Copernicus S3 Product Notice – Altimetry, July 2022, available at https://sentinel.esa.int/documents/247904/2753172/Sentinel-3-Product-Notice-LAND-L1-L2-NRT-STC-NTC.pdf (URL resource last viewed 17^th August 2023)

Global Climate Observing System (2016). The Global Observing System for Climate: Implementation Needs, GCOS-200, available at https://gcos.wmo.int/index.php/en/publications/gcos-implementation-plan2016 (URL resource last viewed 17^th August 2023)

Shepherd et al. (2019). Trends in Antarctic ice sheet elevation and mass. Geophysical Research Letters. https://doi.org/10.1029/2019GL082182  https://doi.org/10.1029/2019GL082182 

Simonsen, Sebastian B., and Louise Sandberg Sørensen. (2017). Implications of Changing Scattering Properties on Greenland Ice Sheet Volume Change from Cryosat-2 Altimetry. Remote Sensing of Environment. https://doi.org/10.1016/j.rse.2016.12.012 (URL resource last viewed 17^th August 2023)

Smith, B., B. P. Jelley, S. Dickinson, T. Sutterley, T. A. Neumann, K. Harbeck. (2021a). ATLAS/ICESat-2 L3B Gridded Antarctic and Arctic Land Ice Height Change, Version 1. Boulder, Colorado USA. NASA National Snow and Ice Data Center Distributed Active Archive Center. https://doi.org/10.5067/ATLAS/ATL15.001 (URL resource last viewed 17^th August 2023)

Smith, B. (2021b). ICESat-2 Algorithm Theoretical Basis Document for Land Ice DEM and Land Ice Height Change Release 001 Algorithm Theoretical Basis Document (ATBD) for Land-ice DEM (ATL14) and Land-ice height change (ATL15). https://icesat-2.gsfc.nasa.gov/sites/default/files/page_files/ICESat2_ATL14_ATL15_ATBD_r001.pdf (URL resource last viewed 17^th August 2023)

Studinger, M. (2014). IceBridge ATM L4 Surface Elevation Rate of Change, Version 299 1, Antarctica subset. N. S. a. I. D. C. D. A. A. Center. Boulder, Colorado, USA. DOI: 10.5067/BCW6CI3TXOCY

5. Related articles

• Page:
  ERB TSI: Algorithm Theoretical Basis Document (ATBD)
• Page:
  ERB TSI: Product User Guide and Specification (PUGS)
• Page:
  IV data version 1.3: System Quality Assurance Document (SQAD)
• Page:
  IV data version 1.3: Product Quality Assurance Document (PQAD)
• Page:
  IV data version 1.3: Product Quality Assessment Report (PQAR)