Contributors: J. Wuite (ENVEO IT GmbH), T. Nagler (ENVEO IT GmbH)
Issued by: ENVEO/ Jan Wuite
Date:
Ref: C3S2_312a_Lot4.WP3-SQAD-IS-v2_202401_IV_System_Quality_Assurance_i1.1
Official reference number service contract: 2021/C3S_312a_Lot4_EODC/SC1
History of modifications
List of datasets covered by this document
Related documents
Acronyms
General definitions
Climate Data Record (CDR): A time series of measurements of sufficient length, consistency, and continuity to determine climate variability and change.
Ice Velocity (IV): Ice flow velocity describes the rate and direction of ice movement. It is a fundamental parameter to characterize the behaviour of a glacier or an ice sheet. Ice velocity and its spatial derivative, strain rate (which is a measure of the ice deformation rate), are required for estimating ice discharge and mass balance and are essential input for glacier models that try to quantify ice dynamical processes.
Offset Tracking (OT): OT refers to several related methods that include speckle tracking, coherence tracking and amplitude tracking or feature tracking. Feature tracking uses cross-correlation of image patches to find the displacement of surface features such as crevasses or rifts and edges, that move with the same speed as the ice and are identifiable on two co-registered amplitude images, to derive ice flow velocity. In coherence tracking, the offset which maximises the interferometric coherence within a certain window size is determined and used to derive the ice velocity. Speckle tracking uses the cross-correlation function of radar speckle patterns, rather than visible features, to derive ice flow velocity.
Scope of the document
This document is the System Quality Assurance Document for the Copernicus Climate Change Service (C3S) Ice Velocity (IV) Climate Data Record (CDR) as part of the Ice Sheets and Ice Shelves service. We document here the production and support systems including system elements, hardware, upgrade cycle, system maintenance and user support for Ice Velocity produced by the service.
Executive summary
This document describes the computing resources and processes involved in making the Ice Velocity data products for Greenland and Antarctica.
Chapter 1 provides a high level description of main components of the processing system (ENVEO Software Package (ESP v2.1). The section gives an overview of satellite and auxiliary data inputs, processing chains, modules, for product generation and validation. It details the input Synthetic Aperture Radar (SAR) data, the validation procedure used for quality control, and the auxiliary datasets needed during the processing. High-level descriptions and a flowchart of the processing chains are given. The output products are annually averaged ice velocity maps covering the Greenland and Antarctic ice sheets, distributed in NetCDF4 format through the Copernicus Climate Change Service (C3S) Climate Data Store (CDS) with yearly updates. Each product covers the period from 1st October to 30th September (Greenland) or 1st April to 31st March (Antarctica), roughly mimicking a glaciological Surface Mass Balance (SMB) year.
In Chapter 2 the computing resources for development and operational processing of the IV products are specified.
The upgrade cycle implemetation procedure is detailed in Chapter 3. Processing of individual velocity maps is done continuously. Annual maps, provided for C3S, are compiled once per year by averaging all ice velocity maps acquired during the (glaciological) year. The compilation of the annual maps starts in November and takes a few weeks including quality control.
Chapter 4 describes reprocessing procedures for CRDs. Reprocessing of the data is only done if necessary (in case of reprocessed Sentinel-1 data) but is generally not foreseen.
Chapter 5 describes data storage and computing system backups, and mitigation plans in the event of systems failure.
Chapter 6 provides information on the Product user support. This is provided through a dedicated contact address available on the Copernicus User Support JIRA Service Desk System.
1. System overview
1.1. System elements and interfaces
In this section, we describe the main components of the processing chain for deriving ice velocity (IV) maps from repeat pass Synthetic Aperture Radar (SAR) data applying offset tracking techniques, including how the interfaces to C3S, and to ancillary data providers are set-up. The Ice Sheet Velocity CDR is produced by ENVEO. The primary processor for IV generation is the ENVEO Software Package (ESP v2.1). Figure 1.1 shows the high-level processing line for the IV production. The system includes 3 main modules:
- IV Module: within this module SAR data and orbit data are imported into the system and velocity maps are generated for pairs of repeat pass data of the same track. The output is a time series of ice velocity maps in map projection. Primary input data are Sentinel-1 Level-1 Single Look Complex (SLC) data acquired in the Interferometric Wide (IW) swath mode. Sentinel-1 data is acquired from the Copernicus Data Space Ecosystem. The Copernicus Data Space Ecosystem maintains an archive of all products (2014-present) and new acquisitions are uploaded within hours. Precise orbits are provided ~3 weeks after acquisition. Auxiliary data includes static digital elevation models (DEMs), which are required for co-registration, calculation of the vertical component of velocity (vz) and final geocoding of the product. The extent of the DEMs is equal to that of the IV products. Land/ocean masks, based on SAR and optical imagery, are used for product finalisation and are updated annually.
- Merge Module: this module combines all IV products from all tracks and all 6 and 12-day repeat image pairs over a specified time span (i.e. 1 year), applying a weighted average approach. The outputs are merged ice velocity maps for all velocity components in x, y and z direction of the map projection (vx, vy, vz and horizontal velocity magnitude) as well as a quality/error map (based on the standard deviation) and valid pixel count map.
- Validation Module: this module facilitates the quality assessment of the IV products, by automating various standard validation tests, including internal consistency checks (for example stable rock test, i.e. check how the algorithm performs on bedrock outcrops where velocity is zero), intercomparisons with ground-based data (in-situ Global Positioning System (GPS)), higher resolution sensors (TerraSAR-X) and independently published datasets. The output is statistical information on the intercomparisons compiled in the Product Quality Assessment Report (D3).
The final output is an annually averaged ice velocity map, distributed in NetCDF4 format through the C3S Climate Data Store (CDS) on a yearly basis.
Figure 1.1: High-level flow chart of the IV processing system. Green 
– input data, Blue 
– processing modules, Red 
- product and intermediate products, Pale Yellow 
– product data base. C3S CDS: Copernicus Climate Change Service Climate Data Store.
1.2. Hardware, supercomputers and cloud computing
Separate systems are used for development and for final operational processing. The main IV processing is done on three server machines and 22 virtual machines on a cluster, which are connected to a mass storage of about 1.5 PB. The system applies Open Multi-Processing (OpenMP) to support multiple Central Processing Units (CPUs) and cores. Development, quality control and product finalisation are done on a server at ENVEO. Table 1.1 provides an overview of the main hardware components.
Table 1.1: Processing hardware for ice velocity at ENVEO.
Development | Processing | |
|---|---|---|
Platform type | Iron Server | High-Performance Cluster |
OS | GNU/Linux Fedora | GNU/Linux Centos 7 |
Number of WS/nodes | 8 | 3 (+22 nodes) |
Processor | Intel Core i7-2600 CPU @ 3.40GHz | Intel Xeon CPU E5-2650 v2 @ 2.60GHz 16 cores |
Memory (RAM) | 32 GB | 128 GB (nodes: 10GB each) |
Local Hard Drive | 1 TB | 300 GB |
Network | Ethernet 1000baseT/Full | Ethernet 10000baseT/Full |
Network Attached Storage | Ca 800 TB network storage | Ca 800 TB network storage |
2. Upgrade cycle implementation procedure
Sentinel-1 data are downloaded on a daily basis, as soon as the product files become available, usually within a few hours after acquisition. Based on new input, processing jobs are created and added to the processing queue automatically once per week. Processing is done continuously. The annual maps, provided for C3S, are compiled once per year by averaging all ice velocity maps acquired during the (glaciological) year. The compilation of the annual map starts in November and takes a few weeks including quality control.
3. Procedures for reprocessing CDR's
Reprocessing of the data is only done if necessary (in case of reprocessed Sentinel-1 data) but is not foreseen.
4. System maintenance and system failures
The software for the processing framework is in an open source version control system capable of running on different systems. Backup of processed data is done on a monthly basis. For all processing nodes, we have a backup of the system image. System maintenance is done when new software distributions become available. As the IV map is produced on an annual basis, production of the IV map is not affected and there is no need to inform users.
5. User support
The Ice Sheets and Ice Shelves service has a team account with the Copernicus User Support (CUS) Jira Service Desk System, to provide level 2 user support, i.e. to answer enquiries specific to their products, by direct interaction with the user through the Jira helpdesk. The request to the CUS can be sent via tag "Scientific or technical question about our products (create ticket)" on the ECMWF Support Portal.
Once a request is sent, the Copernicus User Support Service team at the European Centre for Medium-Range Weather Forecasts (ECMWF) will handle the requests within 8 hours (level 1).
For any scientific and special enquiries that cannot be answered by the CUS team at ECMWF or addressed to the Knowledge Base, the request will be forwarded to the Copernicus User Support Specialists (level-2).
Enquiries forwarded to the Copernicus User Support Specialist team will be acknowledged within 3 working days (target 100%) and a notification sent to the user. In case of specific scientific issues, the enquiries will be channelled to the Essential Climate Variable (ECV) and data specialist of the C3S2_312a_Lot4 project and should be resolved within 3 working weeks (target 85%). In each quarter, we aim for User Support satisfaction scoring 3 in 90% of all voluntary based feedbacks by users, with 1 (very unsatisfied) to 5 (very satisfied). We will also list the number of tickets in the Quarterly Report.
All products made by the team, including the IV product, are handled by the same system.
