Last modified on Jun 27, 2024 17:31

Contributors:  Lin Gilbert (University of Leeds), Sebastian B. Simonsen (Technical University of Denmark), Jan Wuite (ENVEO IT GmbH)

Issued by: University of Leeds / Lin Gilbert

Issued Date: 31/05/2020

Ref:  C3S_312b_Lot4_D1.IS.1-v2.0_202001_System_Quality_Assurance_Document_v1.0

Official reference number service contract:  2018/C3S_312b_Lot4_EODC/SC2 

Note: This document provides the following three deliverables:

    D1.IS.1-v2.0 System Quality Assurance Document - Ice Velocity

    D1.IS.3-v2.0 System Quality Assurance Document – Gravimetric Mass Balance

    D1.IS.5-v2.0 System Quality Assurance Document – Surface Elevation Change

History of modifications

List of datasets covered by this document

Related documents

Acronyms

Scope of the document

This document is the System Quality Assurance Document for the Copernicus Ice Sheets and Ice Shelves service. It describes the organisation of the data processing system and how product updates are implemented.

Executive summary

The service addresses three essential climate variables (ECVs) by providing four separate products.

• Ice velocity is given for Greenland in product D3.IS.4
• Gravimetric mass balance is given for Greenland and Antarctica in product D3.IS.5
• Surface elevation change is given for
  • Antarctica in product D3.IS.6.1
  • Greenland in product D3.IS.6.2

We document the production and support systems for the CDR v2 for each dataset produced by the service. The brokered dataset, gravimetric mass balance, has been previously documented. (Forsberg, 2015)

1. System overview

1.1. System elements and interfaces

1.1.1. Greenland ice sheet velocity – D3.IS.4

In this section, we describe the main components of the processing chain for deriving ice velocity (IV) maps from repeat pass SAR data applying offset tracking techniques, including how the interfaces to C3S, and to the RD component, as well as to ancillary data providers are set-up. The Greenland Ice Sheet velocity CDR is produced by ENVEO It GmbH¹. The primary processor for IV generation is the ENVEO software package (ESP v2.1). Figure 1 shows the high-level processing line for the IV production. The system includes 3 main modules:

• IV Module: within this module SAR data and orbit data are imported into the system and velocity maps are generated for pairs of repeat pass data of the same track. The output is a time series of ice velocity maps in map projection. Primary input data are Sentinel-1A and -1B Level-1 Single Look Complex (SLC) data acquired in the Interferometric Wide (IW) swath mode. ESA provides Sentinel-1 data free of charge through the Copernicus Open Access Hub (SciHub) and various mirror sites. The SciHub site maintains an archive of all products (2014-present) and new acquisitions are uploaded within hours. National mirror sites (e.g. in Austria maintained by ZAMG) provide faster download capabilities, but work on a rolling archive basis, necessitating a local archive. Precise orbits are provided ~ 3 weeks after acquisition. Auxiliary data includes a static DEM, which is required for co-registration, calculation of the vertical component of velocity (v_z) and final geocoding of the product (TanDEM-X 90m DEM; Rizolli et al., 2017). The extent of the DEM is equal to the IV product. Land/ocean masks, based on optical imagery, are used for product finalisation and are updated as required.
• MODULE MERGE: this module combines all IV products from all tracks and all 6 and 12-day repeat image pairs over a specified time span (i.e. 1 year), applying a weighted average approach. The outputs are merged ice velocity maps for all velocity components in x, y and z direction of the map projection (v_x, v_y, v_z and horizontal velocity magnitude) as well as a quality/error map (based on the standard deviation) and valid pixel count map.
• MODULE VAL: this module facilitates the quality assessment of the IV products, by automating various standard validation tests, including internal consistency checks (for example stable rock test, i.e. check how the algorithm performs on bedrock outcrops where velocity is zero), intercomparisons with ground-based data (in-situ GPS), higher resolution sensors (TerraSAR-X) and independently published datasets (MEaSUREs project, available through NSIDC). The output is statistical information on the intercomparisons compiled in a quality assessment report.

The final output is an annually averaged Greenland Ice Sheet velocity map, distributed in NetCDF4 format through the C3S Climate Data Store (CDS) as well as the ENVEO Cryoportal online data portal on a yearly basis.

Figure 1: High-level flow chart of the IV processing system. Green – input data, Blue – processing modules, Red - product and intermediate products, Pale Yellow – product data base.

1.1.2. Gravimetric mass balance – D3.IS.5

The gravimetric mass balance is brokered from the Antarctica and Greenland CCI projects. The processing system consists of one ingestion chain which downloads the two datasets from:

• Antarctica: https://data1.geo.tu-dresden.de/ais_gmb/
• Greenland: http://products.esa-icesheets-cci.org/products/downloadlist/GMB/

After the download of the two individual datasets, the data are combined into a common NetCDF-file, which is then available for the end-users of the C3S products.

1.1.3. The project websites are https:// esa-icesheets-antarctica-cci.org and https:// esa-icesheets-greenland-cci.org, which both have public documentation sections that include their product user guides, algorithm theoretical basis documents and other useful information. Surface elevation change, Antarctica – D3.IS.6.1

The processing system consists of one ingestion chain per input dataset and one chain to combine the ingested data into the final product. Because the input data comes from five different satellites, one of which (ERS1) had two distinct orbital phases that have to be treated separately, the first part of the processing chain has to be repeated for each input dataset. The current input datasets are ERS1 Phase C Reaper L2, ERS1 Phase G Reaper L2, ERS2 Reaper L2, Envisat GDR v2.1, CryoSat-2 L2i baseline C and Sentinel-3A L2 data to the product. A flowchart of the processing chains is shown in Figure 2 below.

Since ERS1, ERS2 and Envisat are no longer operating, and CryoSat-2 baseline C has ceased production (baseline D will be incorporated into the v3 product if it becomes available in time) their ingestion chains only have to be run once. The Sentinel-3A ingestion chain is run monthly, followed by the final product chain. Each ingestion chain follows the same overall scheme but has settings appropriate to the input dataset (e.g. the reference cycle used is different for each mission, a patch must be applied to the Envisat GDR_v2.1 dry tropospheric correction). The ingestion chain accumulates data, changes the tide corrections from the onboard values to a consistent correction obtained from the Circum-Antarctic Tidal Simulator (CATS) 2008a model, calculates the crossover surface elevation changes with reference to a given cycle, applies corrections for glacial isostatic adjustment (GIA) and assembles the dataset into a stacked-grid format. Auxiliary datasets used are the MODIS/ICESat Antarctic surface type mask (Zwally, 2012), the Antarctic slope map (Slater et al, 2018). Auxiliary models used are the IJ05 GIA corrections (Ivins et al, 2005) and the CATS 2008a tide model (Padman et al, 2002). At the end of this part of the chain an intermediate, single-mission, dataset is archived. The combination part of the chain assembles the four intermediate datasets into a single dataset, performing cross-calibration between satellites, then derives rates of surface elevation change, and finally derives key performance indicators before product output. The performance indicators are the product accuracy, stability, geographical coverage and percentage of datapoints within a given uncertainty. There is also a validation chain, run yearly, comparing our surface elevation change rates to those from Operation IceBridge (see Studinger 2014). This is described more fully in the related Product Quality Assurance Document (PQAD).

Figure 2: High-level flow chart of the Antarctic surface elevation rate processing chains. Left: Single mission processing, middle: combined mission processing, right: validation.

All input and auxiliary datasets and models necessary are freely available, although for some registration is required. For more details, including web addresses, see section 3.2 of the related Algorithm Theoretical Basis Document (ATBD).

It should be noted that all the necessary data, except for the most recent data collected by Sentinel-3A, has already been assembled on the University of Leeds system. Sentinel-3A data is downloaded to the system on a regular basis. However, upgrades to all of the input satellite datasets are in progress, and can be incorporated later.

1.1.4. Surface elevation change, Greenland – D3.IS.6.2

As for the Antarctica surface elevation change processor, the Greenland surface elevation processor system consists of one ingestion chain for the older satellites (ERS-1, ERS-2 and ENVISat) and one for the newer satellites (CryoSat-2, and Sentinel-3). These different data ingestion methods are then combined into the final product (see Figure 3).

The current input datasets are ERS1 Phase C Reaper L2, ERS1 Phase G Reaper L2, ERS2 Reaper L2, ENVISat GDR v2.1, CryoSat-2 L2i baseline C, and Sentinel-3A Baseline 003. Further, the decommissioning of ERS1, ERS2 and ENVISat resulted in their ingestion chains only having to be run once. The processing of ERS1, ERS2 and ENVISat relies on the optimal combination of the cross-over (XO) and repeat-track (RT). The novel altimeter onboard CryoSat-2 and Sentinel-3 allows for the surface elevation change to be calculated by plan-fitting algorithm. The ingestion chain for will run monthly as long as CryoSat-2 and/or Sentinel-3 is operational, followed by the final product chain. For more details, including web addresses, see section 4.2 of the related Algorithm Theoretical Basis Document (ATBD).

At the end of satellite dependent processing chains an intermediate, single-mission, dataset is archived. The final processing chain assembles the five intermediate datasets into a single dataset, performing cross-calibration between satellites, then deriving rates of surface elevation change, and finally deriving key performance indicators before product output. The performance indicators are the product accuracy, stability, geographical coverage and percentage of data-points within a given uncertainty. Following these performance indicators, the validation chain is run annually and compares the derived surface elevation change rates to those from Operation IceBridge (see Studinger 2014). This procedure is described in more detail in the related Product Quality Assurance Document (PQAD).

All ESA radar altimetry data is downloaded to the storage servers at the Technical university of Denmark on a daily basis. The data-storage forms the basis of the operational system developed within ESA's Climate Change Initiative and is also the backbone of the process in the Ice Sheets and Ice Shelves service of the C3S Land Hydrology and Cryosphere project. However, upgrades to all of the main satellite datasets (ERS1 Phase C Reaper L2, ERS1 Phase G Reaper L2, ERS2 Reaper L2, ENVISat GDR v2.1, and CryoSat-2 L2i baseline C) are in progress, and the system is ready for accommodating the updates, with the expectance of minor unforeseen challenges. All input and auxiliary datasets and models necessary are freely available, although for some registration is required. For more details, including web addresses, see the related Algorithm Theoretical Basis Document, Section 4.2.

Figure 3: High-level flow chart of the Greenland surface elevation rate processing chains. Left: Older mission processing, middle: the flow chart for the newer satellites as exemplified by CryoSat-2 and the combined mission processing, right: validation.

1.2. Hardware, supercomputers and cloud computing

1.2.1. Greenland ice sheet velocity – D3.IS.4

The main IV processing is done on three server machines and 18 virtual machines at the IKB cluster, which are connected to a mass storage of about 800 TB. The system applies OPENMP to support multiple CPUs and cores. Development, quality control and product finalisation are done on a server at ENVEO. Table 1 provides an overview of the main hardware components.

Table 1: Processing hardware for Greenland Ice Sheet IV at ENVEO.

1.2.2. Gravimetric mass balance – D3.IS.5

Not applicable for the brokered GMB data, as the creation of the final NetCDF-product is done in a python-environment on a laptop.

1.2.3. Surface elevation change, Antarctica – D3.IS.6A

The team has access to a wide range of computing resources at the University of Leeds comprising a mixture of centrally-run University Facilities and more specialist Faculty systems supporting around 200Tb of storage. The University has a High-Performance Computing system with a configuration of around 2000 cores based on Intel Nehalem processors.

The Centre for Polar Observation and Monitoring (CPOM) at the University of Leeds has in the past year established a presence on the Joint Analysis System Meeting Infrastructure Needs (JASMIN) facility, which provides infrastructure for environmental data analysis across the United Kingdom. It is operated by the Centre for Environmental Data Analysis (CEDA), funded by the UK's Natural Environment Research Council (NERC). This resource is available to the C3S team.

1.2.4. Surface elevation change, Greenland – D3.IS.6.2

The team at DTU Space have access to a scalable range of computing facilities. A High-Performance cluster with 48 treads has been chosen for the SEC processing. All development has been done on a macOS system with 4 cores and limited amount of memory and storage capabilities.

2. Upgrade cycle implementation procedure

2.1. Greenland ice sheet velocity – D3.IS.4

Sentinel-1 data are downloaded on a daily basis, as soon as the product files become available, usually within a few hours after acquisition. Based on new input, processing jobs are created and added to the processing queue automatically once per week. Processing is done continuously. The annual maps, provided for C3S, are compiled once per year by averaging all (6/12-d repeat) IV maps and run from 1st October to 30 September, roughly mimicking a glaciological SMB year. The compilation of the annual map starts in November and takes a few weeks including quality control.

2.2. Gravimetric mass balance – D3.IS.5

Not applicable for the brokered GMB data. This is a complete, finished product from the Gravity Recovery and Climate Experiment (GRACE) mission. Currently the processing of the GRACE-FO (GRACE Follow-On) mission is still in the R&D stage, and when this processing has matured, it will be added.

2.3. Surface elevation change, Antarctica – D3.IS.6.1

Sentinel-3A level 2 data is usually available for download approximately 35 days after it is acquired. The system at the University of Leeds checks for new data once per day and downloads whatever is available. Crossover processing requires whole cycles (or pseudo-cycles) of data, which in the case of Sentinel-3A are 27 days long. The processing chain is then started. From data ingestion to the product being made available, it takes approximately 3 days to run. In practice the time lag between data acquisition and product update is about 2 months.

The validation dataset is updated irregularly, but usually yearly, and the validation chain run once it becomes available. The validation chain only takes a few hours to run.

2.4. Surface elevation change, Greenland – D3.IS.6.2

Both Sentinel-3 and CryoSat-2 level-2 data are usually available for download approximately 35 days after they are acquired by the satellite. The system setup at the Technical University of Denmark downloads all new data daily, and the Greenland surface elevation change processors are run every month by an automated processing procedure. When the automated processing is done, a processing summary is sent to all relevant persons at DTU space. When the processing e-mail is received, the final product undergoes human inspection before being pushed to an ftp-site, where the product is released for the CDS. In practice, this will result in a time lag between data acquisition and product update of about 2 months. The validation dataset is updated irregularly, but usually yearly as a result of spring field season, and the validation chain run once it becomes available. The validation chain only takes a few hours to run. We foresee the migration to ingestion of ICESat-2 data in the validation effort in the future, and when this is implemented, we can perform the validation effort more frequently.

3. Procedures for reprocessing CDR's

3.1. Greenland ice sheet velocity – D3.IS.4

Reprocessing of the data is only done if necessary (in case of reprocessed Sentinel-1 data) but is not foreseen.

3.2. Gravimetric mass balance – D3.IS.5

The GMB have been processed once for the initial data release, and new data releases are monitored, and will be incorporated annually. The GRACE science mission ended in 2017, and only limited updates are expected. The GRACE-FO mission processing still requires a significant R&D effort. We follow the R&D intensively and as soon we see promising result the brokering of GRACE-FO GMB will be implemented, but at time of writing no schedule for data release is available.

3.3. Surface elevation change, Antarctica – D3.IS.6.1

Reprocessing is expected as part of the programme of yearly improvements to the processing system. The CDR v2 contains all of the data from the v1 CDR, with the addition of data from Sentinel-3A, processed using a new cross-calibration method to retrieve extra data. The Envisat dataset currently in use, GDR v2.1, has been upgraded to GDR v3.0, and will be used in the v3 CDR. The Envisat single-mission chain will be run to update the basic data, and then the v3 combined mission chain will incorporate it. Similarly, an upgraded Sentinel-3A dataset, with better handling of marginal land ice, is expected in mid-2021. It may be issued too late to be included in this project. The CryoSat-2 dataset currently in use, baseline C, will be upgraded to baseline D in the future. It is expected to be ready this year, and if so can be incorporated into the v3 CDR. A project to reprocess the Reaper dataset from ERS1 and ERS2 is planned, but no data availability date has been set. For details, please see related document, the Development Milestone Plan.

The validation dataset, IceBridge ATM L4 Surface Elevation Rate of Change V001 from https://icebridge.gsfc.nasa.gov, is also updated yearly, but irregularly, with flight campaigns in the Arctic and Antarctic spring. Validation is performed on a yearly basis to accommodate this schedule, using the full dataset from the project start. This necessitates the re-running of the validation chain from the beginning.

3.4. Surface elevation change, Greenland – D3.IS.6.2

See Section 3.3 above. Both Greenland and Antarctic SEC use the same input data and have the same processing schedule.

4. System maintenance and system failures

4.1. Greenland ice sheet velocity – D3.IS.4

The software for the processing framework is in an open source version control system capable of running on different systems. Backup of processed data is done on a monthly basis. For all processing nodes, we have a backup of the system image. System maintenance is done when new software distributions become available. As the Greenland Ice Sheet IV map is produced on an annual basis, production of the IV map is not affected and there is no need to inform users.

4.2. Gravimetric mass balance – D3.IS.5

Not applicable for this brokered dataset.

4.3. Surface elevation change, Antarctica – D3.IS.6.1

The University of Leeds system is fully backed up once per week, with incremental backups on the other 6 days. Backups are held at Leeds and also replicated off-site, and a monthly backup on physical media is also held off-site. System maintenance is kept to short periods and advanced warning is given, so processing can be planned around known outages.

Further, the code needed to run the full processing chain is written in C, IDL and shell scripts, all of which can be implemented on many different replacement systems in the unlikely event of a major failure.

The product will be archived at the University of Leeds, but copies will be pushed to the Earth Observation data Centre (EODC), who will supply them to the Climate Data Store (CDS). Users can only obtain data via the CDS. Thus, failure of the University of Leeds system will not affect the availability of any product already made. In unlikely circumstances failure could delay production of further ICDRs, in which case the CDS will be informed.

4.4. Surface elevation change, Greenland – D3.IS.6.2

The system at DTU Space undergoes an incremental backup each day and a full back-up 4 times a year, both on physical media. System maintenance is kept to short periods and advanced warning is given. Such system maintenance cycles will not affect the users of the product as the monthly product update cycle can be planned around known system outages. If needed the processing chain can be run on alternative servers, as the code needed to run the full processing chain is written in python 3.5, allowing the system to run on any replacement systems, which is found suitable in the unlikely event of a major failure. The product will be archived at the Technical University of Denmark, but copies will be pushed to the Climate Data Store (CDS). Users can only obtain data via the CDS. Thus, failure of the Technical University of Denmark system will not affect the availability of any product already made. In unlikely circumstances failure could delay production of further ICDRs, in which case the CDS will be informed.

5. User support

5.1. Greenland ice sheet velocity – D3.IS.4

The IV processing team at ENVEO has an account with the Copernicus User Support JIRA Service Desk System, to handle level 2 user enquiries through the JIRA helpdesk. For IV the user support is provided by:
Contact Person Name: Jan Wuite
E-mail Address: c3s-support@enveo.at

5.2. Gravimetric mass balance – D3.IS.5

As in section 5.4, below.

5.3. Surface elevation change, Antarctica – D3.IS.6.1

The AIS SEC team has a team account with the Copernicus User Support Jira Service Desk System, to provide level 2 user support, i.e. to answer enquiries specific to the product, by direct interaction with the user through the Jira helpdesk. The team contact list is held by the University of Leeds.
E-mail Address: cpom@leeds.ac.uk

5.4. Surface elevation change, Greenland – D3.IS.6.2

The GrIS SEC and GMB processing team at DTU Space has an account with the Copernicus User Support Jira Service Desk System, to handle level 2 user enquiries through the Jira helpdesk. The user support is provided by:
Contact Person Name: Sebastian Simonsen
E-mail Address: c3s-support@space.dtu.dk

References

Forsberg, R. et al, 2015, System Specification Document (SSD) for the Greenland Ice Sheet CCI Project of ESA's Climate Change Initiative, version 1.0. Available from http://www.esa-icesheets-greenland-cci.org/

Ivins, E. R. and James, T. S., (2005). Antarctic glacial isostatic adjustment: a new assessment. Antarctic Science, 17(4), 541-553

Padman, L., et al (2002). A new tide model for the Antarctic ice shelves and seas. Annals of Glaciology, 34, 247-254

Rizzoli, P., Martone, M., Gonzalez, C., Wecklich, C., Borla Tridon, D., Bräutigam, B., Bachmann, M., Schulze, D., Fritz, T., Huber, M., Wessel, B., Krieger, G., Zink, M., and Moreira, A. (2017): Generation and performance assessment of the global TanDEM-X digital elevation model. ISPRS Journal of Photogrammetry and Remote Sensing, Vol 132, pp. 119-139.

Slater, T., et al, (2018). A new digital elevation model of Antarctica derived from CryoSat-2 altimetry, The Cryosphere, 12, 1551-1562, doi: 10.5194/tc-12-1551-2018

Studinger, M., 2014, updated 2018. IceBridge ATM L4 Surface elevation rate of change, version 1. Boulder, Colorado, USA. NASA Snow and Ice Data Center Distributed Active Archive Centre. DOI: 10.5067/BCW6CI3TXOCY

Zwally, H. Jay, Mario B. Giovinetto, Matthew A. Beckley, and Jack L. Saba, 2012, Antarctic and Greenland Drainage Systems, GSFC Cryospheric Sciences Laboratory at http://icesat4.gsfc.nasa.gov/cryo_data/ant_grn_drainage_systems.php

6. Related articles

• Page:
  IV Version 1.2; SEC Version 2.0; GMB Version 2.0: Product User Guide Specification (PUGS)
• Page:
  IV Version 1.2; SEC Version 2.0; GMB Version 2.0: System Quality Assurance Document (SQAD)
• Page:
  IV Version 1.2; SEC Version 2.0; GMB Version 2.0: Product Quality Assurance Document (PQAD)
• Page:
  IV Version 1.2; SEC Version 2.0; GMB Version 2.0: Product Quality Assessment Report (PQAR)
• Page:
  IV Version 1.2; SEC Version 2.0; GMB Version 2.0: Algorithm Theoretical Basis Document (ATBD)