Contributors: Lin Gilbert (University of Leeds), Sebastian B. Simonsen (Technical University of Denmark), Jan Wuite (ENVEO IT GmbH)
Issued by: University of Leeds / Lin Gilbert
Date: 25/06/2021
Ref:C3S_312b_Lot4_D2.IS.6-v3.0_202103_PQAR_SEC_i1.0.docx
Official reference number service contract: 2018/C3S_312b_Lot4_EODC/SC2
History of modifications
List of datasets covered by this document
Related documents
Acronyms
Scope of the document
This document is the Product Quality Assessment Report for the Copernicus Ice Sheets and Ice Shelves Service, surface elevation change ECV. It presents results of the quality assessment for the provided Antarctic and Greenland datasets and a discussion of how well Global Climate Observing System (GCOS) and user requirements have been met.
Executive summary
The service addresses three essential climate variables (ECVs) by providing four separate products.
- Ice velocity is given for Greenland in product D3.IS.4
- Gravimetric mass balance is given for Greenland and Antarctica in product D3.IS.5
- Surface elevation change is given for
- Antarctica in product D3.IS.6.1
- Greenland in product D3.IS.6.2
In this document we only provide the results from the CDR v3.0 for the surface elevation change datasets produced by the service.
1. Product validation methodology
1.1. Surface elevation change, Antarctica
The product is validated against data provided by the Airborne Topographic Mapper (ATM), a scanning laser altimeter flown on board aircraft by Operation IceBridge (Studinger 2014). A comparison is made between the level 4 product, IceBridge ATM L4 Surface Elevation Rate of Change V001 and a dataset matching its measurements in location and time, calculated from the Antarctic surface elevation change product's underlying data. For further details see the related Product Quality Assurance Document where both the IceBridge dataset is described and the validation methodology.
As well as validation against external data, to comply with user requirements gathered by the Global Climate Observing System (GCOS) the product should achieve two statistical targets, and the C3S project has identified two key performance indicators, as shown in Table 1.
Table 1: Antarctic surface elevation change product targets
Statistic | Target | Target source |
Stability at pixel-level | 0.1 m/y | GCOS |
Accuracy at basin-level | 0.1 m/y | GCOS |
Accuracy at pixel-level | 0.1 m/y | C3S project |
Surface coverage, aggregated over 1 year | 65% ERS1, ERS2, Envisat, Sentinel3-A and Sentinel3-B | C3S project |
The stability is taken as one standard deviation of the linear model fit to the surface elevation change timeseries used in deriving the surface elevation change rate.
The accuracy is the total error budget for the surface elevation change rate, which has three components - the input uncertainty, the uncertainty due to cross-calibration (if applicable) and the stability, as summarized below.
The input uncertainty is the standard deviation of the surface elevation change values within the aggregation area, ie within a single pixel or within a basin.
The uncertainty due to cross-calibration is the standard deviation of the cross-calibration bias estimate. The cross-calibration method uses multiple regression, using the REGRESS function in the IDL v8 software package. This provides uncertainty estimates for each coefficient in the regression, given input uncertainties as described above.
The three components are summed in quadrature to produce the total error. This method is applicable at both basin and pixel level.
The product only supplies pixel level data. It can be used to make basin level datasets, but the method employed should be selected by the user to best match their needs. The basin level datasets used to check against the targets are made with a simple methodology, where data gaps in any basin at a given time are filled by an ice-velocity-guided value derived from the rest of the basin, using the Berkeley – Ice Sheet Initiative for Climate Extremes (BISICLES) ice velocity model. This, along with a more detailed discussion of the uncertainty processing is discussed in the Algorithm Theoretical Basis Document.
1.2. Surface elevation change, Greenland
In line with the Antarctica surface elevation changes, the Greenland counterpart is also validated against data provided by NASA's Operation IceBridge (OIB) Airborne Topographic Mapper (ATM), which is a scanning laser altimeter (Studinger 2014). The ATM instrument has been flown in Greenland since 1993, and within the OIB data-package are estimates of surface change from all repeat measurements of surface elevation done by OIB. This higher-level product (level-4) has been downloaded from NSIDC (https://nsidc.org/icebridge/portal/map
). In validation of the surface elevation change (SEC), we use the derived parameter of the cumulative surface elevation change (dh), as this product can be used to derive SEC averaging of the same time span as the time span in-between repeat observations by the OIB ATM. As the OIB data usually are obtained once-a-year (in the spring), we here summarize the mean and standard deviation of the difference in SEC between OIB and our SEC. This mean and standard deviation of the estimate difference is ascribed to the year of the first ATM observation.
As well as validation against external data, to comply with user requirements gathered by the Global Climate Observing System (GCOS) the product should achieve two statistical targets, and the C3S project has identified two KPIs, as shown in Table 1 for Antarctica. The Greenland surface elevation change also uses the KPIs outlined in Table 1, except for the surface coverage which due to more optimal location of the Greenland ice sheet in relation to the satellite obits and differences in postprocessing interpolation becomes obsolete.
Here, the stability is taken as one standard deviation of the linear model fit to the surface elevation change timeseries used in deriving the surface elevation change rate. Whereas the accuracy is the total error budget for the surface elevation change rate. This, along with a more detailed discussion of the uncertainty processing is discussed in the Algorithm Theoretical Basis Document.
2. Validation results
2.1. Surface elevation change, Antarctica
Figure 1 shows the IceBridge comparison results. All IceBridge Antarctic data available from the portal, as of 31st January 2021, was considered. However, for operation reasons, IceBridge flights were concentrated on the West Antarctic Ice Sheet, Filchner-Ronne Ice Shelf and surrounding regions. In total 222732 points of comparison, covering 443 grid cells, were used. Figure 1 shows the locations for which comparison was possible, a scattergram of the values compared and a histogram of their differences. In general the points of the scattergram lie along or close to the line of equivalence, where Y = X, as expected. The cell-averaged outlier-resistant mean difference (ie excluding outliers of more than 3 sigma) was 0.027 ± 0.650 m/yr, with a correlation coefficient of 0.58. Although no specific validation target figures were given, this is within the GCOS surface elevation change target accuracy of 0.1 m/yr.
Figure 1: AIS SEC Comparison to IceBridge. Left to right - comparison locations, scattergram of surface elevation change rates for the same locations from IceBridge and the Antarctic SEC products, and histogram of surface elevation change rate differences between the two products.
The major evolution of the SEC dataset between v2 and v3 was the inclusion of improved mission datasets from EnviSat (GDR v2.1 replaced by GDR v3) and CryoSat-2 (baseline C replaced by baseline D). A large proportion of the Operation IceBridge data covers the time period used by these two missions, ie from mid-2002 onwards. As can be seen by comparison to the v2 PQAR, the centre of the cell-averaged resistant mean difference distribution has improved from 0.081 to 0.027 m/yr, while the overall distribution remains similar.
Figure 2 shows the statistical results. The histograms of dataset accuracy show both component and total figures. The three components are;
- epoch uncertainty - the uncertainty at each measurement epoch, which combines all sources of error from the input data
- cross-calibration uncertainty – the uncertainty in the biasing applied to the timeseries' from each mission to produce a final, continuous multi-mission timeseries
- model uncertainty – the uncertainty in the fitting of the linear model used to derive the SEC, which is also the measure of the data stability
It can be seen in Figure 2 that the dominant uncertainty comes from the altimetry measurements (the epoch uncertainty) – its distribution has the least percentage of instances within the GCOS target, and its peak is at the highest value. At pixel level the observations are closely clustered, but at basin level they incorporate a wide range of terrain and thus the overall uncertainty is higher.
Coverage changes depend on altimetry mission. ERS1, ERS2, Envisat, Sentinel3-A and Sentinel3-B are unable to observe within 8.5° of the South Pole, so 20% of the Antarctic Ice Sheet cannot be covered by these missions. However, they all follow repeating orbits, which allows for regular crossover analysis across the observed regions. CryoSat-2 can observe within 2° of the pole, excluding only 1% of the AIS, but has a drifting orbit on a very long repeat cycle. This makes crossover analysis more challenging, especially in coastal regions where surface slope is high and altimetric measurements are not always possible.
It should be noted that the current release of Sentinel3-A and B data has known problems when the satellite track crosses from ocean to land, leading to loss of data in coastal regions. Because of this the Sentinel3-A and B datasets have less coverage than expected. This problem will be fixed with a new land-ice specific processor which will be used to reprocess all of the mission data, but unfortunately the reprocessed data will not be released until mid-2021, which is too late for its inclusion in the v3 product. See ESA Mission Management July 2020, section #S3-1.
Figure 2: Statistical summary from Antarctic surface elevation change product. Top to bottom - pixel-level accuracy, basin-level accuracy and percentage coverage of the Antarctic ice sheets and shelves region. The accuracy statistics are shown by total and component, and the percentage of instances within the GCOS target are given.
2.2. Surface elevation change, Greenland
Figure 3 shows the OIB ATM intercomparison results. Here, all Greenland OIB elevation change data available from the NISDC, as of January 2021, was considered. The figure shows the geolocated difference between the OIB and C3S observations (right panel), the point-to-point intercomparison and the distribution of the differences. The version 3 solution shows improvements in the overall fitting stability and many of the outliers seen in version 2 have been removed. The majority of the OIB observations are located at the fast-changing outlet glaciers, which bias the statistics as the radar observations applied in the C3S elevation change estimates struggles to resolve the short length-scales at which the elevation change is occurring at these outlet glaciers. This gives rise to the broader distribution towards negative values in the lower-left panel of Figure 3. With the version 3 updates, we have lowered the overall bias by about 1 cm per year and removed a large part of the outliers seen in the fitting stability in version 2. Hence, the median difference (bias) for version 3 is -1.25 cm per year.
Figure 3: (Right) The spatial distribution of the point-to-point inter-comparison with the OIB ATM surface elevation estimates. (Left-upper) The point-to-point correlations between the C3S-surface elevation change and the OIB surface elevation estimate. (Left-lower) The histogram of the point-to-point differences between C3S and OIB surface elevation change. For reference, the C3S v.2 estimate is also included in the left panel
Additionally, Figure 4 shows the statistics from the internal-fitting accuracy, here we see an improvement with the update of version 3. This seemingly small improvement needs to be put into context of the increased accuracy of more than 25% from version 1 to version 2 of the products. .
Figure 4: The stability in the linear model fit to the surface elevation change timeseries used in deriving the surface elevation change rate. For reference, the C3S v.2 estimate is also included. As seen are the v.3 providing a clear improvement in relation to the GCOS requirements.
3. Application(s) specific assessments
3.1. Surface elevation change, Antarctica – D3.IS.6.1
Not applicable.
3.2. Surface elevation change, Greenland – D3.IS.6.2
Not applicable
4. Compliance with user requirements
4.1. Surface elevation change, Antarctica – D3.IS.6.1
In validation, the cell-averaged mean difference (excluding outliers beyond 3 standard deviations) between the independent and C3S datasets was 0.027 ± 0.650 m/yr. This is within the GCOS target accuracy of 0.1 m/yr.
In accuracy, at pixel level the C3S target accuracy of 0.1 m/yr is reached or bettered in approximately one third of all instances, with the peak of the distribution lower than the target, at 0.075m. At basin level the additional uncertainty from the larger collection area and data gap filling means that the majority of instances do not reach the GCOS target. The distribution is wide and flat, and peaks close to the target at 0.155m (Figure 2).
In stability, the GCOS target of 0.1 m/yr (see Table 1) is reached or bettered in approximately 79% of all instances.
In coverage, the C3S target of 65% is generally achieved during the ERS2 and Envisat mission periods, but during the ERS1 period the coverage is generally 20% lower due to sparse altimetric data. During the CryoSat-2 mission period the C3S target of 90% is never achieved, in fact only approximately half of that is observed in any given month due to the drifting orbit as described previously in section 2.1. As also described in that section, Sentinel3-A and B coverage is less than it should be, which will be fixed in the next data reprocessing. However, since CryoSat-2 and Sentinels 3A and 3B are currently operating simultaneously, an improvement in coverage to just below 80% is visible at the present time (Figure 2).
4.2. Surface elevation change, Greenland – D3.IS.6.2
The uncertainty given in the product is the epoch uncertainty (derived from the supplied input data) and the model uncertainty (derived from the plane-fitting) combined. The evaluation of the uncertainty estimates is shown in figures 3 and 4. As seen, the number of points with an accuracy within the GCOS requirements are 99.3%. This estimate is purely an internal stability indication and not the real error estimate, as multiple factors are not included in this estimate. Factors such as changes in penetration depth of the radar, slope induced relocation errors and that the radar only observers changes at the highest point within its footprint, which may bias the elevation change estimate. Therefore, the real error estimate and the number which needs to fulfill the user-requirements needs are to be found by applying independent validation of the surface elevation change product. Here, the independent validation is provided by comparison to NASA's OIB airborne laser altimetry campaigns. Figure 3 shows the result of the inter-comparison between the OIB and C3S-Greenland SEC, with a mean bias of all observations of 1.25 cm per year. Thereby, the current processing version fulfils the GCOS requirement (given in Table 1).
References
Barletta, V.R., Sørensen, L.S. and Forsberg, R. (2013). Scatter of mass changes estimates at basin scale for Greenland and Antarctica. The Cryosphere, 7, 1411-1432
ESA Mission Management, Copernicus S3 Product Notice – Altimetry, July 2020, available at https://sentinel.esa.int/documents/247904/2753172/Sentinel-3-Product-Notice-STM-Level-2-Land
Forsberg, R. et al.(2017) User Requirements Document (URD), ESA Climate Change Initiative (CCI), Greenland Ice Sheet (GIS) Essential Climate Variable (ECV), Version, 2.4, 22-11-2017, ST-DTU-ESA-GISCCI-URD-001, available at: http://esa-icesheets-greenland-cci.org/index.php?q=webfm_send/169
Forsberg, R. et al. (2018) Product Validation and Intercomparison Report (PVIR), ESA Climate Change Initiative (CCI), Greenland Ice Sheet (GIS) Essential Climate Variable (ECV), Version, 3.0, 25-06-2018, ST-DTU-ESA-GISCCI-PVIR-001, available at: http://esa-icesheets-greenland-cci.org/index.php?q=webfm_send/180
Hvidberg, C.S., et al., User Requirements Document for the Ice_Sheets_cci project of ESA's Climate Change Initiative, version 1.5, 03 Aug 2012. Available from: http://www.esa-icesheets-cci.org/
Studinger, M. (2014). IceBridge ATM L4 Surface Elevation Rate of Change, Version 299 1, Antarctica subset. N. S. a. I. D. C. D. A. A. Center. Boulder, Colorado, USA. DOI: 10.5067/BCW6CI3TXOCY
This document has been produced in the context of the Copernicus Climate Change Service (C3S).
The activities leading to these results have been contracted by the European Centre for Medium-Range Weather Forecasts, operator of C3S on behalf of the European Union (Delegation agreement signed on 11/11/2014). All information in this document is provided "as is" and no guarantee or warranty is given that the information is fit for any particular purpose.
The users thereof use the information at their sole risk and liability. For the avoidance of all doubt , the European Commission and the European Centre for Medium - Range Weather Forecasts have no liability in respect of this document, which is merely representing the author's view.
