Last modified on Nov 28, 2023 17:06

Contributors: Jacqueline Bannwart (University of Zurich), Inés Dussailant (University of Zurich), Frank Paul (University of Zurich), Michael Zemp (University of Zurich)

Issued by: UZH / Frank Paul

Date: 18/07/2023

Ref: C3S2_312a_Lot4.WP2-FDDP-GL-v1_202212_A_PQAR-v4_i1.1

Official reference number service contract: 2021/C3S2_312a_Lot4_EODC/SC1

History of modifications

List of datasets covered by this document

Related documents

Acronyms

General definitions

For the glacier area product we define the Randolph Glacier Inventory (RGI) as a Climate Data Record (CDR) and the regionally and temporarily constrained improvements or updates submitted to the Global Land Ice Measurements from Space (GLIMS) database as Interim Climate Data Records (ICDRs). The latter might be integrated in new releases of the CDR. This document is related to the generated ICDRs.

Debris-cover: Debris on a glacier is usually composed of unsorted rock fragments with highly variable grain size (from mm to several m). These might cover the ice in lines of variable width separating ice with origin in different accumulation regions of a glacier (so called medial moraines) up to a complete coverage of the ablation region. Automated mapping of glacier ice is only possible when the debris is not covering the ice completely in regard to the image pixel size.

Digitizing uncertainty: The standard deviation of the glacier area differences resulting from independent multiple digitizing of the same glacier outline by the same analyst. This value usually increases towards smaller glaciers.

Interpretation uncertainty: The standard deviation of the glacier area differences resulting from independent digitizing of the same glacier outlines by at least two different analysts. This value usually increases with the number of difficult glaciers (e.g. debris-covered) in the sample.

Glacier area: The area (or size) of a glacier, usually given in the unit km². Also used by Global Climate Observing System (GCOS) to name the related Essential Climate Variable (ECV) product.

Glacier outline: A vector dataset with polygon topology marking the boundary of a glacier.

Glacier inventory: A compilation of glacier outlines with associated attribute information.

Scope of the document

This document is the Product Quality Assesment Report (PQAR) for the Copernicus glacier distribution service providing results of the quality assessment for the glacier area product generated by the glacier distribution service of the Copernicus Climate Change Service (C3S).

The related datasets are regionally constrained (Interim Climate Data Records; ICDRs) for the forthcoming Randolph Glacier Inventory 7 (RGI7), which is a global dataset of glacier outlines that will be available as a Climate Data Record (CDR) from the Climate Data Store (CDS). The focus is on a quantification of the achieved quality improvements compared to the outlines currently available in RGI6. An overall assessment of RGI quality based on Pfeffer et al. (2014) is provided in the Product Quality Assurance Document (PQAD) [RD1].

Executive summary

The datasets for which we here present a quality assessment have been largely created earlier by other analysts. The methodological approach we use here to determine their quality is thus not based on multiple digitizing or the buffer method, but a direct comparison of the differences. We thus present first the difference between digitizing and interpretation uncertainty to explain why we here only analyse the latter. We then describe details of the method (addition of two gridded versions to obtain omission and commission errors) and present in Section 2 the results of the comparison. Differences between the old (RGI6) and the new datasets created by us are mostly based on including too much seasonal snow, missed glaciers, wrong interpretation of regions in shadow or under debris cover, and temporal changes, i.e. the glacier outlines were simply out-dated. In the last Section we evaluate the results compared to Global Climate Observing System (GCOS) requirements, concluding that they provide a clear improvement in regard to the most important aspects for RGI7 (better quality and closer to the year 2000).

1. Product validation methodology

1.1. Background

The methods available for determination of product uncertainty are detailed in Section 3 of the PQAD [RD1] and are thus not repeated here. Due to rarely available validation data, we usually determine product uncertainty rather than systematic and random errors. When creating a fresh dataset (any information linked to the production of the data is presented in the ATBD [RD2]), uncertainty of the outlines can best be determined by independent multiple digitizing of the same glaciers. Overlay of resulting outlines and calculation of the standard deviation of the area differences are the measures to visualize and calculate the digitizing uncertainty (e.g. Paul et al. 2013 and 2020). This uncertainty is usually a few per-cent of the glacier area and much smaller than the interpretation uncertainty, which occurs when different analysts are digitizing the same glaciers. Here, much larger differences in interpretation can occur, e.g. for debris-covered glaciers or misinterpreted seasonal snow.

For the three datasets provided to GLIMS for RGI7 during C3S2 (see Section 2), we reinterpreted already existing datasets, i.e. we have to consider the larger interpretation uncertainty. This was achieved by calculating the area covered by omission and commission errors, add them up and compare them to the new and the original extent. In other words, we determine (a) the area mapped by both analysts, (b) the area missed in RGI6 (e.g. very small, strongly shaded or debris-covered glaciers) and (c) the area that was too large in RGI6 (e.g. due to wrongly mapped seasonal snow). The obtained (relative) change in area might be used as a measure for the improvement in a region, but we do not recommend this as the value strongly depends on the total area of the corrected dataset. Instead, we also present overlays of the previous and new datasets to discuss the reasons for the differences. These are not always related to poor mapping quality.

1.2. Approach

To determine the regions for (a), (b) and (c), we convert the outlines of both datasets (from RGI6 and our improved one) to a regular grid using vector-raster conversion and assign a value of 0 to regions without glaciers in both datasets, a value of 1 to glaciers in RGI6 and a value of 2 to glaciers in our improved dataset. A simple addition of both grids gives 0 for no glaciers in both datasets, 1 for glaciers only in RGI6 (too large), 2 for glaciers only present in our improved dataset (i.e. missed in RGI6) and 3 for glaciers in both datasets. The pixel counts for each of the values 1, 2 and 3 are given in the value attribute table of the respective raster file. These are used to determine the relative portions of each class and, by multiplication with the grid cell area, the area in km². Comparison with the original or the new size gives relative area changes due to the corrections.

2. Validation results

2.1. Kenai Peninsula (Alaska)

A new dataset of glacier extents from 2005 for the Kenai Peninsula was provided by Yang et al. (2020). A request if this dataset would supersede the quality of the existing outlines in RGI6 provided by Kienholz et al. (2015) and should thus be used to replace it, resulted in a check of the new outlines by the C3S team. The comparison revealed that both the RGI6 and the new dataset by Yang et al. suffered from including too much seasonal snow, excluding some debris covered as well as small glaciers and having drainage divides at wrong locations. We thus decided to revise the RGI6 outlines using the RGI6 outlines as a starting point and the Yang et al. dataset as a guide. Most of the work was related to removing wrongly mapped snow cover (Figure 1). The pixel counts (and related areas) are summarized in Table 1 for the added, deleted and common pixels along with sums, differences and percentages per category.

Figure 1: Overlay of classified glacier grids for a sub-region of the Kenai Peninsula. RGI6 includes the grey and red pixels, the new dataset the grey and blue ones, i.e. regions in red were deleted, those in blue added. Image width is 46.65 km, North is at top.

Table 1: Results of the glacier map comparison for Kenai Peninsula. The column 'Both' gives the area that agrees in both datasets and column 'Sum N+D' the sum of the 'New' and 'Deleted' pixels. 'Total old' is the sum of 'Both' and 'Deleted', whereas 'Total new' is 'Both' + 'New'. The last column is their difference.

Overall, the glacier area in this region was 1.1% too large in RGI6 with the sum of the new and deleted pixels being 3.75% of the former area. This comparably small difference is due to the large area covered by glaciers and because most of the edits concern comparably small glaciers.

2.2. Brooks Range (Alaska)

For the Brooks Range, a 'new' glacier was reported that did not exist in RGI6. This lead to (a) a check if further glaciers are 'missing' in this mountain range and (b) to a comparison of existing outlines with very high-resolution satellite images as well as Landsat images. This revealed that a larger number (>50) of further glaciers, most of them very small, were also missing, that some outlines were shifted and some extents were too large. The latter probably due to a wrong interpretation of snow or debris cover. We thus reworked the dataset using the outlines in RGI6 from Kienholz et al. (2015) as a starting point and two Landsat 7 scenes (from 2000 and 2005) for a new interpretation. The resulting overlay of the two glacier grids is shown in Figure 2. Table 2 is presenting the differences between the two datasets.

Figure 2: Overlay of classified glacier grids for a sub-region of the Kenai Peninsula. RGI6 includes the grey and red pixels, the new dataset the grey and blue ones, i.e. regions in red were deleted, those in blue added. Image width is 29.5 km, North is at top.

Table 2: Results of the glacier map comparison for Brooks Range. The column 'Both' gives the area that agrees in both datasets and column 'Sum N+D' the sum of the 'New' and 'Deleted' pixels. 'Total old' is the sum of 'Both' and 'Deleted', whereas 'Total new' is 'Both' + 'New'. The last column is their difference.

Overall, the glacier area in this region was 7.3% too small in RGI6 with the sum of new and deleted pixels being 16.9% of the former area. This comparably large difference is mostly due to the many unconsidered glaciers or glacier parts and because of a small shift in geolocation.

2.3. Kerguelen Island (Sub-antarctica)

Glacier outlines for Kerguelen Island did not have a date in RGI6 and might refer to a topographic map from 1963 (Berthier et al. 2009). They were thus outdated and partly also generalized. A new submission to GLIMS by E. Berthier (IDs 518 and 527) provided updated outlines for the largest part of the island. Unfortunately, the south-western part was missing in this submission to GLIMS and many of the smaller glaciers south of the Cook Ice Cap where too large due to seasonal snow. We used the GLIMS outlines submitted by E. Berthier as a starting point and a series of Landsat and Sentinel-2 images to properly identify and map the glaciers in the southwest.

Figure 3: Overlay of classified glacier grids for Kerguelen Island. RGI6 includes the grey and red pixels, the new dataset the grey and blue ones, i.e. regions in red were deleted, those in blue added. Image width is 60 km, North is at top.

Table 3: Results of the glacier map comparison for Kenai Peninsula. The column 'Both' gives the area that agrees in both datasets and column 'Sum N+D' the sum of the 'New' and 'Deleted' pixels. 'Total old' is the sum of 'Both' and 'Deleted', whereas 'Total new' is 'Both' + 'New'. The last column is their difference.

For the largest and second largest ice caps, the comparison with RGI6 in Figure 3 mainly shows the retreat of outlet glaciers between the 1960s and 2000. Between these two ice caps, some glaciers were missing and others have been very roughly digitized. The smallest ice cap in the southwest might have been surrounded by seasonal snow during the mapping. For the glaciers in the southeast (Massif Gallieni) we see some shrinkage, a reinterpretation of the large debris-covered glacier (Buffon) in the southeast and geolocation issues, i.e. a rotation of some glaciers in the southwest to the north. The strong area reduction of nearly 27% is thus a consequence of both glacier shrinkage and improved interpretation. We need to add that glacier outlines for the largest ice cap (Cook) as provided by E. Berthier were only marginally adjusted.

3. Application(s) specific assessments

We did not perform any further glacier or application specific assessments.

4. Compliance with user requirements

Basic user requirements for RGI7 are to have quality improved glacier outlines that are acquired closer to the year 2000. The technical requirements given in the latest GCOS Implementation Plan (GCOS 2022) still demand an area uncertainty that is smaller than 5% (see [RD3] for details). This can usually only be achieved when each glacier outline is visually checked and corrected if required. The quality assessment of the three examples from RGI6 in Section 2 is related to interpretation uncertainty, i.e. glacier outlines are interpreted by different persons. We here corrected rather typical issues, i.e. too much snow was mapped as glaciers for the Kenai Peninsula, a large number of mostly small glaciers were missing in the Brooks Range and glacier extents were far away from the target year 2000 for Kerguelen Island. We have thus certainly moved these three datasets closer to user and GCOS requirements, but the area changes alone are not very indicative of the improvement. This is largely due to the strong dependence of the changes on the original area, which gives only very small overall changes for Kenai Peninsula. For many individual glaciers the area reduction is larger than 50%. Vice versa, the strong area reduction for the glaciers on Kerguelen Island is largely governed by the timing (strong shrinkage since the 1960s) rather than by the quality of the outlines. However, as the timing is also a user requirement, the new datasets provide a considerable improvement compared to the outlines available in RGI6.

References

Berthier, E., R. Le Bris, L. Mabileau, L. Testut, and F. Rémy (2009). Ice wastage on the Kerguelen Islands (49°S, 69°E) between 1963 and 2006, J. Geophys. Res., 114, F03005; doi: 10.1029/2008JF001192

GCOS (2022): The 2022 GCOS ECVs requiremnts. GCOS-245, published by WMO, pp. 244.

Kienholz, C., S.J. Herreid, J.L. Rich, A. Arendt, R. Hock, E.W. Burgess (2015). Derivation and analysis of a complete modern-date glacier inventory for Alaska and northwest Canada. Journal of Glaciology, 61 (227), 403–420.

Paul, F., N. Barrand, E. Berthier, T. Bolch, K. Casey, H., rey, S.P. Joshi, V. Konovalov, R. Le Bris, N. Mölg, G. Nosenko, C. Nuth, A. Pope, A. Racoviteanu, P. Rastner, B. Raup, K. Scharrer, S. Steffen and S. Winsvold (2013): On the accuracy of glacier outlines derived from remote sensing data. Annals of Glaciology, 54 (63), 171-182.

Paul, F., Rastner, P., Azzoni, R. S., Diolaiuti, G., Fugazza, D., Le Bris, R., ... & Smiraglia, C. (2020). Glacier shrinkage in the Alps continues unabated as revealed by a new glacier inventory from Sentinel-2. Earth System Science Data, 12(3), 1805-1821.

Pfeffer, W. T., Arendt, A. A., Bliss, A., Bolch, T., Cogley, J. G., Gardner, A.S., … Sharp, M. J. (2014). The Randolph Glacier Inventory: a globally complete inventory of glaciers. Journal of Glaciology, 60(221), 537-552. DOI: 10.3189/2014JoG13J176

Yang, R., Hock, R., Kang, S., Shangguan, D. and Guo, W. (2020). Glacier mass and area changes on the Kenai Peninsula, Alaska, 1986–2016. Journal of Glaciology 66 (258), 603–617.

Related articles

• Page:
  ECV - Glaciers, Ice Sheets, Lakes, Soil Moisture
• Page:
  IV Version 1.2; SEC Version 2.0; GMB Version 2.0: Product User Guide Specification (PUGS)
• Page:
  IV Version 1.2; SEC Version 2.0; GMB Version 2.0: System Quality Assurance Document (SQAD)
• Page:
  IV Version 1.2; SEC Version 2.0; GMB Version 2.0: Product Quality Assurance Document (PQAD)
• Page:
  IV Version 1.2; SEC Version 2.0; GMB Version 2.0: Product Quality Assessment Report (PQAR)