Contributors: Lin Gilbert (University of Leeds), Sebastian B. Simonsen (Technical University of Denmark), Jan Wuite (ENVEO IT GmbH)
Issued by: University of Leeds / Lin Gilbert
Date: 27/04/2021
Ref: C3S_312b_Lot4.D1.IS.1-v3.0_IV_System_Quality_Assurance_Document_i1.0
Official reference number service contract: 2018/C3S_312b_Lot4_EODC/SC2
History of modifications
List of datasets covered by this document
Related documents
Acronyms
Scope of the document
This document is the System Quality Assurance Document for Ice Velocity (IV) as part of the Copernicus Ice Sheets and Ice Shelves service. It describes the organisation of the data processing system and how product updates are implemented.
Executive summary
The service addresses three essential climate variables (ECVs) by providing four separate products.
- Ice velocity is given for Greenland in product D3.IS.4
- Gravimetric mass balance is given for Greenland and Antarctica in product D3.IS.5
- Surface elevation change is given for
- Antarctica in product D3.IS.6.1
- Greenland in product D3.IS.6.2
We document here the production and support systems for the CDR v3 for Ice Velocity produced by the service.
1. System overview
1.1. System elements and interfaces
In this section, we describe the main components of the processing chain for deriving ice velocity (IV) maps from repeat pass SAR data applying offset tracking techniques, including how the interfaces to C3S, and to the RD component, as well as to ancillary data providers are set-up. The Greenland Ice Sheet velocity CDR is produced by ENVEO It GmbH1. The primary processor for IV generation is the ENVEO software package (ESP v2.1). Figure 1 shows the high-level processing line for the IV production. The system includes 3 main modules:
- IV Module: within this module SAR data and orbit data are imported into the system and velocity maps are generated for pairs of repeat pass data of the same track. The output is a time series of ice velocity maps in map projection. Primary input data are Sentinel-1A and -1B Level-1 Single Look Complex (SLC) data acquired in the Interferometric Wide (IW) swath mode. ESA provides Sentinel-1 data free of charge through the Copernicus Open Access Hub (SciHub) and various mirror sites. The SciHub site maintains an archive of all products (2014-present) and new acquisitions are uploaded within hours. National mirror sites (e.g. in Austria maintained by ZAMG) provide faster download capabilities, but work on a rolling archive basis, necessitating a local archive. Precise orbits are provided ~3 weeks after acquisition. Auxiliary data includes a static DEM, which is required for co-registration, calculation of the vertical component of velocity (vz) and final geocoding of the product (TanDEM-X 90m DEM; Rizolli et al., 2017). The extent of the DEM is equal to the IV product. Land/ocean masks, based on optical imagery, are used for product finalisation and are updated as required.
- MODULE MERGE: this module combines all IV products from all tracks and all 6 and 12-day repeat image pairs over a specified time span (i.e. 1 year), applying a weighted average approach. The outputs are merged ice velocity maps for all velocity components in x, y and z direction of the map projection (vx, vy, vz and horizontal velocity magnitude) as well as a quality/error map (based on the standard deviation) and valid pixel count map.
- MODULE VAL: this module facilitates the quality assessment of the IV products, by automating various standard validation tests, including internal consistency checks (for example stable rock test, i.e. check how the algorithm performs on bedrock outcrops where velocity is zero), intercomparisons with ground-based data (in-situ GPS), higher resolution sensors (TerraSAR-X) and independently published datasets (MEaSUREs project, available through NSIDC). The output is statistical information on the intercomparisons compiled in a quality assessment report.
The final output is an annually averaged Greenland Ice Sheet velocity map, distributed in NetCDF4 format through the C3S Climate Data Store (CDS) as well as the ENVEO Cryoportal online data portal on a yearly basis.
Figure 1: High-level flow chart of the IV processing system. Green – input data, Blue – processing modules, Red - product and intermediate products, Pale Yellow – product data base.
1.2. Hardware, supercomputers and cloud computing
The main IV processing is done on three server machines and 18 virtual machines on a cluster, which are connected to a mass storage of about 800 TB. The system applies OPENMP to support multiple CPUs and cores. Development, quality control and product finalisation are done on a server at ENVEO. Table 1 provides an overview of the main hardware components.
Table 1: Processing hardware for Greenland Ice Sheet IV at ENVEO.
Development | Processing | |
|---|---|---|
Platform type | Iron Server | High-Performance Cluster |
OS | GNU/Linux Fedora | GNU/Linux Centos 7 |
Number of WS/nodes | 8 | 3 |
Processor | Intel Core i7-2600 CPU @ 3.40GHz | Intel Xeon CPU E5-2650 v2 @ 2.60GHz 16 cores |
Memory (RAM) | 16 GB | 128 GB |
Local Hard Drive | 1 TB | 300 GB |
Network | Ethernet 1000baseT/Full | Ethernet 10000baseT/Full |
Network Attached Storage | Ca 800 TB network storage | Ca 800 TB network storage |
2. Upgrade cycle implementation procedure
Sentinel-1 data are downloaded on a daily basis, as soon as the product files become available, usually within a few hours after acquisition. Based on new input, processing jobs are created and added to the processing queue automatically once per week. Processing is done continuously. The annual maps, provided for C3S, are compiled once per year by averaging all (6/12-d repeat) IV maps and run from 1st October to 30 September, roughly mimicking a glaciological SMB year. The compilation of the annual map starts in November and takes a few weeks including quality control.
3. Procedures for reprocessing CDR's
Reprocessing of the data is only done if necessary (in case of reprocessed Sentinel-1 data) but is not foreseen.
4. System maintenance and system failures
The software for the processing framework is in an open source version control system capable of running on different systems. Backup of processed data is done on a monthly basis. For all processing nodes, we have a backup of the system image. System maintenance is done when new software distributions become available. As the Greenland Ice Sheet IV map is produced on an annual basis, production of the IV map is not affected and there is no need to inform users.
5. User support
The IV processing team at ENVEO has an account with the Copernicus User Support JIRA Service Desk System, to handle level 2 user enquiries through the JIRA helpdesk. For IV the user support is provided by:
Contact Person Name: Jan Wuite
E-mail Address: c3s-support@enveo.at
References
Rizzoli, P., Martone, M., Gonzalez, C., Wecklich, C., Borla Tridon, D., Bräutigam, B., Bachmann, M., Schulze, D., Fritz, T., Huber, M., Wessel, B., Krieger, G., Zink, M., and Moreira, A. (2017): Generation and performance assessment of the global TanDEM-X digital elevation model. ISPRS Journal of Photogrammetry and Remote Sensing, Vol 132, pp. 119-139.
This document has been produced in the context of the Copernicus Climate Change Service (C3S).
The activities leading to these results have been contracted by the European Centre for Medium-Range Weather Forecasts, operator of C3S on behalf of the European Union (Delegation agreement signed on 11/11/2014). All information in this document is provided "as is" and no guarantee or warranty is given that the information is fit for any particular purpose.
The users thereof use the information at their sole risk and liability. For the avoidance of all doubt , the European Commission and the European Centre for Medium - Range Weather Forecasts have no liability in respect of this document, which is merely representing the author's view.
