Last modified on Jul 01, 2026 08:12

Contributors: F. Paul (University of Zurich), M. Zemp (University of Zurich), J. Bannwart (University of Zurich)

Issued by: University of Zurich / Frank Paul

Date: 30 Apr 2026 

Ref: C3S2_WP1-DDP-GA-01_202511_PQAR_v7.0

Official reference number service contract: 2024/C3S2_313d_ENVEO/SC1 

History of modifications

List of datasets covered by this document

Acronyms

General definitions

For the glacier area product we define the Randolph Glacier Inventory (RGI) as a Climate Data Record (CDR) and the regionally and temporarily constrained improvements or updates submitted to the Global Land Ice Measurements from Space (GLIMS) database as Interim Climate Data Records (ICDRs). The latter might be integrated in new releases of the CDR. This document is related to the generated ICDRs.

Debris-cover: Debris on a glacier is usually composed of unsorted rock fragments with highly variable grain size (from mm to several m). These might cover the ice in lines of variable width separating ice with origin in different accumulation regions of a glacier (so called medial moraines) up to a complete coverage of the ablation region. Automated mapping of glacier ice is only possible when the debris is not covering the ice completely in regard to the image pixel size.

Digitizing uncertainty: The standard deviation of the glacier area differences resulting from independent multiple digitizing of the same glacier outline by the same analyst. This value usually increases towards smaller glaciers.

Interpretation uncertainty: The standard deviation of the glacier area differences resulting from independent digitizing of the same glacier outlines by at least two different analysts. This value usually increases with the number of difficult glaciers (e.g. debris-covered) in the sample.

Glacier area: The area (or size) of a glacier, usually given in the unit km². Also used by Global Climate Observing System (GCOS) to name the related Essential Climate Variable (ECV).

Glacier outline: A vector dataset with polygon topology marking the boundary of a glacier.

Glacier inventory: A compilation of glacier outlines with associated attribute information.

Executive summary

This document is the Product Quality Assessment Report (PQAR) for the Copernicus glacier service, providing results of the quality assessment for the glacier area product generated for the Copernicus Climate Change Service (C3S). The assessment presented in this document refers to a regionally constrained subset of glacier outlines for Baffin Island. This is an Interim Climate Data Record (ICDR) that is not available from the Climate Data Store (CDS), but provided to the GLIMS glacier database for open availability and long-term storage.

In this document we first present an overview of the error sources and uncertainties introduced during data processing before the method to determine uncertainty as applied here is described. As the main uncertainty for the dataset is related to the manual correction of the raw glacier outlines, we perform a classical multiple digitizing experiment to determine the digitizing uncertainty of the analyst. Results are presented in the form of outline overlays, a table with all data and a scatterplot showing the dependency of the uncertainty on glacier area. For the full sample of 15 glaciers an average uncertainty (defined as the standard deviation of the digitized areas) of 4.5% was derived. Thereby, the digitizing on 10 m resolution Sentinel-2 images has a nearly three times lower uncertainty (2.2%) than with the 30 m Landsat images (6.0%). First results of a climate change assessment are also presented, revealing strong glacier shrinkage on Baffin Island from 2000 to 2019 for glaciers <10 km² (-1.3% a^-1). We have as yet no application specific assessments of the dataset, but confirm that the average uncertainty obtained for the test site is smaller than the 5% breakthrough value of GCOS and thus compliant with the requirements.

1. Product validation methodology

1.1. Background

Due to rarely available validation data (e.g. orthorectified higher-resolution datasets acquired in the same week as the satellite image), glacier outlines are usually not validated against a reference dataset but the uncertainty of the glacier area is determined in relative terms by independent multiple digitizing of the same glaciers. Overlay of resulting outlines and calculation of the standard deviation of the area differences are the measures to be applied. Whereas the overlay of the resulting outlines reveal the critical regions (with large variability) of the digitizing, the uncertainty derived from the standard deviation of the area differences is a quantitative measure that can be used to determine the significance of glacier area changes. Below, we provide a description of the uncertainties for each processing step.

As detailed in the ATBD (Paul et al. 2024), the method to create glacier outlines from optical satellite images has a pre-, main- and post-processing part. By selecting the best cloud-free images from the end of the ablation period (i.e. with a minimum amount of seasonal snow) at the pre-processing stage, two major sources of error are largely excluded: (a) cloud cover (that will lead to a systematic underestimation of glacier extents) and (b) seasonal snow (that will lead to a systematic overestimation). Although both errors can be largely corrected during the post-processing stage, high uncertainties would remain if images with better conditions are not available. Another error to be considered is (c) the geolocation error, caused by errors in the Digital Elevation Model (DEM) and/or the Ground Control Points (GCPs) used by satellite data providers for orthorectification of the raw images (Landsat L1T and Sentinel-2 L1C products). Whereas this error has a limited direct impact on the derived glacier outlines, it has a few indirect impacts: it limits the possibilities to use other geolocated datasets for correcting the outlines (e.g. very high-resolution images available via a web map service, WMS) or (ii) it creates ice divides at the wrong location (e.g. mountain crests in the DEM cut through glaciers) causing too large and too small glaciers. The shifts in geolocation can be corrected if they are systematic (e.g. Nuth and Kääb 2011), but not if they are random, i.e. differ from pixel to pixel due to DEM errors.

In the main processing step, the selected threshold applied to the band ratio image decides about the work remaining for correcting mapping errors of ice in shadow. Due to highly variable illumination conditions (e.g. from bright snow on opposite rock walls) a single threshold that fully includes ice in shadow on one mountain slope might also include bare rock in shadow on another mountain slope. Hence, the threshold is always a compromise and should be tailored to reduce the workload for manual corrections in the post-processing stage. Independent of the selected threshold value, debris-covered ice is not included and has to be corrected anyway. Water surfaces (rivers, lakes, ocean) are partly also wrongly mapped and might be removed by classifying it with a threshold applied to another band (e.g. for ocean water) or after raster-vector conversion at the post-processing stage (for rivers and lakes). In particular glacier fronts calving into lakes with a high turbidity are usually connected to the lakes in the raw classification.

After the classified glacier map has been converted to vector outlines, the editing in the post-processing stage is usually divided into a first and quick removal of commission errors (e.g. wrongly classified rivers, lakes, sea ice, ice clouds) and a more demanding adjustment of omission errors (e.g. debris cover, ice in shadow). The latter has a topological part related to the correct connection of all individual polygons (usually separated by the not included debris bands, i.e. medial moraines) that belong to a single glacier and the removal of isolated holes (due to debris) within a glacier polygon. This is usually done by digitizing a rough polygon that includes everything (the holes and the individual polygons) and the sub-sequent merging of all parts.

The second part to the correction of omission errors is the correction of the outline itself using the original satellite image and possibly very high-resolution images and already existing glacier outlines as a guide for the interpretation and digitizing. The latter two impact on the uncertainties of the final product. As the manual digitizing generalizes the outline across many pixels, this is never done at exactly the same position when performed twice. The digitizing uncertainty can thus be determined by an independent digitization (at least three times) of a couple of glaciers (10 to 20) with varying sizes and difficulties (debris, shadow) by calculating the resulting mean area of a glacier and the standard deviation of the area values in relative terms (e.g. Paul et al. 2013). This value strongly increases towards smaller glaciers and slightly decreases towards higher spatial resolution of the satellite image.

The interpretation uncertainty is related to the glacier definition applied (the rule-set for the parts to be considered as a glacier) and varies with the analyst. Largest differences are usually found for debris-covered ice with a gradual transition to ice-cored moraines and rock glaciers as well as dead ice (dynamically disconnected from the main glacier) and features such as snow-covered perennial ice patches where it is unclear if ice is underneath or not (and whether or not these features should be included). To a larger extent this uncertainty also depends on the spatial resolution, i.e. when more details become visible towards higher resolutions, the decisions on the outline location can get more complex and thus inconsistent. This uncertainty can be determined by a digitizing experiment with several analysts, each doing it independently at least three times for 5 to 10 different glaciers (e.g. Paul et al. 2013 and 2020).

1.2. Approach

In this phase of C3S2 we have created a new glacier inventory from August 2019 for Baffin Island (from 2020 for the glaciers on the Hall and Meta Incognita Peninsula) using nine Landsat 8/9 and three Sentinel-2 scenes. As all outlines have been created from scratch (with the methods described in the ATBD, Paul et al. 2024) and most of them had to be manually corrected due to very dark (and thus not automatically mapped) outer perimeters as well as debris cover, we have performed uncertainty assessment with the classical multiple digitizing approach described above. In total, 15 glaciers of different sizes (0.1 to 5.7 km²) and with different difficulties (debris, shadow, dark perimeters) were selected and digitized independently three times. The location of these glaciers is depicted in Figure 1 along with their numbering for later reference. Nine glaciers are selected from Landsat 8 scene 015-013 (path-row) acquired on 12.08.2019 (Figure 1a) and six are part of Sentinel-2 tile 19WER acquired on 13.08.2019 (Figure 1b). The outlines from the original dataset were shortly analyzed before the digitizing to avoid gross interpretation errors but turned off before the digitizing. Very high-resolution images provided by the ESRI Basemap (World Imagery Layer) and the Google Earth WMS were also made visible for interpretation and in a few cases also for the digitizing (see examples in Section 2).

Following the three digitizing rounds, glacier area was calculated (in the local UTM projection) for each of the fifteen glaciers. These values were stored along with the area mapped during the original digitizing (as stored in the inventory). The mean area and standard deviation (relative to the mean area) was calculated for each glacier and summarized for all four outlines (incl. the original one) and only the three outlines from the multiple digitizing experiment. This allows a separation of the digitizing uncertainty provided by the experiment from the interpretation uncertainty that is larger and applies when also the original digitizing is considered. Values were calculated separately for the Landsat and Sentinel-2 digitizing and also calculated for all 15 glaciers. The average standard deviation of the digitized glacier areas provides the uncertainty for each sample and the total.

Figure 1. Location and numbers of the glaciers selected for uncertainty assessment. a) The nine glaciers on the Landsat scene (numbered and marked in yellow) and b) the same for the six glaciers of the Sentinel-2 scene. The size of the glaciers from the Landsat scene is rather typical for the region whereas those from Sentinel-2 are comparably small. Images: a) earthexplorer.usgs.gov (last access: 27.11.2025), b) Copernicus Sentinel data 2019.

2. Uncertainty assessment

Figures 2, 3 and 4 present overlays of the outlines and the digitized original dataset for selected glaciers in both regions and Table 1 presents a summary of the derived glacier areas along with the mean values and standard deviations. Figure 2a shows the overlay for glaciers 3, 9 and 8 (from left to right) along with the outlines (in light blue) from the 2019 inventory. Some variability in interpretation for the shadow and debris-covered parts is already visible at this scale. Figures 2b and c show a close-up of the terminus section of the largest glacier (Nr. 9) in the image centre at the original 30 m resolution of Landsat in Figure 2b and a very-high-resolution image from the ESRI Basemap for comparison (in Figure 2c). Whereas the boundary of the glacier is barely visible with Landsat, panel 2c reveals the complex structure of the debris covered and retreating terminus. Several of the merged medial moraines are ice cored and still connected to the main glacier, making proper identification challenging. Accordingly and despite the back check with the original interpretation and the use of the very high-resolution image for the digitizing, there is a high variability in the interpretation of this region. However, the deviations are not systematic and the standard deviation of the area differences is only 0.5%. For the two other glaciers (Nr. 3 and 8 in Figure 1a) the outlines are much more consistent.

Figure 2. a) Outline overlay for the Landsat scene with glaciers 3 (upper left), 9 (centre) and 8 (upper right) marked by green numbers. Outlines from the inventory are in light blue, the white, yellow and red outlines are from the digitizing experiment. The green frame marks the region shown in panels b) and c). b) Close-up of the terminus section of glacier 9 with the 30 m resolution Landsat scene in the background and c) with a very high-resolution image from the ESRI Basemap. Even with this image the interpretation of the debris-covered parts is difficult. Images: a), b) earthexplorer.usgs.gov (last access: 27.11.2025); c) ESRI Basemap.

The two glaciers (Nr. 5 and 6 in Figure 1a) in Figure 3 also reveal the highest variability in interpretation for glacier regions in shadow and under debris cover. The close-ups in Figures 3b and 3c as well as 3d to 3f show this variability even more clearly. As before, the deviations are not systematic and area differences are thus small in the mean. For glacier nr. 6 the view with the 15 m panchromatic band (Figure 3e) has been added to reveal how resolution impacts on the visibility of details. In this case there is a gradual transition to an ice-debris landform that shows typical features like water-filled holes and a steep frontal part that would be typical for rock glaciers. Thanks to the very high-resolution image there is at least some consistency in the interpretation. If only the Landsat images were used, the entire landform might have been mapped as a part of the glacier. This might also be correct as the ice below the debris contributes to run off, but it might give wrong results when glacier changes are calculated.

Figure 3. a) Outline overlay for the Landsat scene with glaciers 5 (upper left) and 6 (lower right) marked by green numbers. Outlines from the inventory are in light blue, the white, yellow and red outlines are from the digitizing experiment. The green frame marks the region shown in panels d), e) and f). b) Close-up of glacier 5 with the 30 m resolution Landsat scene in the background and c) with a very high-resolution image from Google Earth. d) Close-up of the terminus section of glacier 6 with the 30 m resolution Landsat image in the background e) as d) but with the 15 m panchromatic band, f) as d) but with a very high-resolution image from Google Earth. Images: a), b), d), e) earthexplorer.usgs.gov (last access:27.11.2025); c), f) Google Earth WMS.

Finally, Figure 4 shows the results with Sentinel-2 for four differently sized ice caps with dirty boundaries and two glaciers with some debris cover and being partly in shadow. For the outline corrections of the four ice caps (Figures 4a to d) very high-resolution images had not been used or were not available, whereas for the two other glaciers (nr. 5 and 6 in Figure 1b) an image available from the WMS of Google Earth was used to correct the parts in shadow (see Figure 4f). For all glaciers some local outline variability in difficult regions can be seen, but they are somewhat smaller than shown for the examples with Landsat.

Figure 4. Outline overlay for the Sentinel-2 scene showing all six glaciers selected for the multiple digitizing (the green numbers refer to Figure 1b). Outlines from the inventory are in light blue, the white, yellow and red outlines are from the digitizing experiment. Panels a) to d) show ice caps with dirty boundaries that were not mapped with the algorithm. Panels e) and f) depict glaciers with shadow and debris cover. Images: a) to e) Copernicus Sentinel Data 2019, f) Google Earth WMS.

The comparison of the glacier areas obtained from the multiple digitizing with the value from the new inventory is presented in Table 1 along with mean values and standard deviations for each glacier as well as all glaciers digitized from Sentinel-2, Landsat 8 and the full sample. A graphical representation of the standard deviation vs. mean glacier size is shown in a scatterplot (Figure 5). The digitizing uncertainty for the full sample is 4.5% and thus below the GCOS limit of 5% for this ECV. However, towards the smaller glaciers in the sample the uncertainty is higher, reaching up to 14%. As expected, the area uncertainty for the Sentinel-2 sample is with 2.2% nearly 3 times lower than for Landsat with 6.0%. This is likely a direct consequence of the three times higher resolution of Sentinel-2. With one exception (glacier nr. 1 in the Landsat sample), glacier areas in the inventory are always a few per cent smaller than from the digitizing experiment (3.1% in total). This indicates a slightly different (more generous) interpretation of the extent for the multiple digitizing than in the original interpretation.

Table 1. Glacier areas for the three rounds of independent digitzing and the inventory along with an average area and standard deviation (Stddev) for each glacier as well as the Sentinel-2, Landsat 8 and full sample. The row 'Sum' is just adding all values of a sample and its standard deviation refers to this sum. The row 'Mean' presents the more relevant average of all individual values (highlighted in red). This is also the case for both samples in the last row.

We have thus separately calculated the standard deviations for the three test digitizations, showing that they are indeed somewhat lower (1.8% for Sentinel-2, 5.1% for Landsat 8, 3.7% for both). This can also be seen for the individual glaciers in Figure 5, where the markers for the sample '(test)' are usually below those from the combined values.

Figure 5. Standard deviations of the digitized glacier areas vs. mean glacier area for individual glaciers and the different samples.

3. Climate Change Assessment

Once available, we will present here a comparison between glacier extents derived for the year 2000 inventory of Baffin Island with the new one created for 2019. First results indicate a strong overall shrinkage of -12.5% (-0.66% a^-1) when excluding Barnes Ice Cap and of -25% (-1.3% a^-1) when only considering glaciers <10 km². This value is the same as in the Alps for about the same period and similarly sized glaciers (Paul et al. 2020). A related overlay of glacier outlines from both inventories is shown in Figure 6.

Figure 6. Overlay of glacier outlines for the year 2000 and 2019 inventories of Baffin Island. The example shows several disintegrating ice caps that will disappear soon. Image: Copernicus Sentinel Data 2019.

4. Application specific assessments 

The new inventory for Baffin Island has been created as a contribution to a forthcoming new version of the RGI that will refer to extents from around the year 2020. No further product specific assessments have yet been performed.

5. Compliance with user requirements concerning data quality

The relevant technical requirements for the glacier area product as given in the latest GCOS Implementation Plan (GCOS 2022) is an area uncertainty that is smaller than 5% ('breakthrough' value) for individual glaciers (see Paul et al. 2023 for details). In regions with debris-covered glaciers, this can only be achieved when each glacier outline is visually checked and corrected if required. The uncertainty derived here for the Baffin 2019 dataset from a multiple digitizing experiment is 4.5% and thus within the acceptable margin. Considering that the area uncertainty is increasing towards smaller glaciers and the glaciers in the sample are comparably small (most of them are smaller than 2.1 km²), the 4.5% area uncertainty derived here is a very good result. Table 2 summarizes the compliance of the generated product with GCOS requirements.

Table 2. Compliance of the glacier area product with GCOS requirements. The shading in the column 'Reported value' indicates which threshold requirement (listed in columns 2, 3, and 4) the dataset meets: Green: "goal", orange: "threshold", yellow: "breakthrough".

References

GCOS (2022): The 2022 GCOS ECVs requiremnts. GCOS-245, published by WMO, pp. 244.

Paul, F., N. Barrand, E. Berthier, T. Bolch, K. Casey, H., rey, S.P. Joshi, V. Konovalov, R. Le Bris, N. Mölg, G. Nosenko, C. Nuth, A. Pope, A. Racoviteanu, P. Rastner, B. Raup, K. Scharrer, S. Steffen and S. Winsvold (2013): On the accuracy of glacier outlines derived from remote sensing data. Annals of Glaciology, 54 (63), 171-182; doi.org/10.3189/2013AoG63A296

Paul, F., Rastner, P., Azzoni, R. S., Diolaiuti, G., Fugazza, D., Le Bris, R., Nemec, J., Rabatel, A., Ramusovic, M., Schwaizer, G.,and Smiraglia, C. (2020). Glacier shrinkage in the Alps continues unabated as revealed by a new glacier inventory from Sentinel-2. Earth System Science Data, 12(3), 1805-1821; doi.org/10.5194/essd-12-1805-2020

Paul, F. et al. (2023): C3S Glacier Area: Target Requirements and Gap Analysis Document (TRGAD). Copernicus Climate Change Service. Document ref. C3S2_312a_Lot4.WP3-TRGAD-GL-v2_202304_A_TR_GA_i1.1

Paul, F. et al. (2024): C3S Glacier Area: Algorithm Theoretical Basis Document (ATBD). Copernicus Climate Change Service. Document ref.
C3S2_312a_Lot4.WP2-FDDP-GL-v2_202312_A_ATBD-v5_i1.0

Related articles

• Page:
  SEC version 5.0: Product Quality Assurance Document (PQAD)
• Page:
  SEC Version 5.0: System Quality Assurance Document (SQAD)
• Page:
  Glacier Area document version 2023: Target Requirements and Gap Analysis Document (TRGAD)
• Page:
  Glacier Area document version 2022: Target Requirements and Gap Analysis Document (TRGAD)
• Page:
  Glaciers area document version 2021: Product Quality Assessment Report (PQAR)