Contributors: Lin Gilbert (University of Leeds), Sebastian B. Simonsen (Technical University of Denmark), Jan Wuite (ENVEO IT GmbH)
Issued by: University of Leeds / Lin Gilbert
Date: 03/05/2021
Ref: C3S_312b_Lot4_D1.IS.5-v3.0_202001_System_Quality_Assurance_Document_SEC_v1.0
Official reference number service contract: 2018/C3S_312b_Lot4_EODC/SC2
History of modifications
List of datasets covered by this document
Related documents
Acronyms
Scope of the document
This document is the System Quality Assurance Document for Surface Elevation Change (SEC) as part of the Copernicus Ice Sheets and Ice Shelves service. It describes the organisation of the data processing system and how product updates are implemented.
Executive summary
The service addresses three essential climate variables (ECVs) by providing four separate products.
- Ice velocity is given for Greenland in product D3.IS.4
- Gravimetric mass balance is given for Greenland and Antarctica in product D3.IS.5
- Surface elevation change is given for:
- Antarctica in product D3.IS.6.1
- Greenland in product D3.IS.6.2
We document here the production and support systems for the CDR v3 for Surface Elevation Change (SEC) for the two polar regions produced by the service.
1. System overview
1.1. System elements and interfaces
1.1.1. Surface elevation change, Antarctica – D3.IS.6.1
The processing system consists of one ingestion chain per input dataset and one chain to combine the ingested data into the final product. Because the input data comes from six different satellites, one of which (ERS1) had two distinct orbital phases that have to be treated separately, the first part of the processing chain has to be repeated for each input dataset. The input datasets used for CDRv3 are ERS1 Phase C Reaper L2, ERS1 Phase G Reaper L2, ERS2 Reaper L2, Envisat GDR v3, CryoSat-2 L2i baseline D, Sentinel-3A L2 and Sentinel-3B L2. A flowchart of the processing chains is shown in Figure 1 below.
Since ERS1, ERS2 and Envisat are no longer operating their ingestion chains only have to be run once. The CryoSat-2 and Sentinel-3 ingestion chains are run monthly, followed by the final product chain. Each ingestion chain follows the same overall scheme but has settings appropriate to the input dataset (e.g. the reference cycle used is different for each mission). The ingestion chain accumulates data, changes the tide corrections from the onboard values to a consistent correction obtained from the Circum-Antarctic Tidal Simulator (CATS) 2008a model, calculates the crossover surface elevation changes with reference to a given cycle, applies corrections for glacial isostatic adjustment (GIA) and assembles the dataset into a stacked-grid format. Auxiliary datasets used are the MODIS/ICESat Antarctic surface type mask (Zwally, 2012), the Antarctic slope map (Slater et al, 2018). Auxiliary models used are the IJ05 GIA corrections (Ivins et al, 2005) and the CATS 2008a tide model (Padman et al, 2002). At the end of this part of the chain an intermediate, single-mission, dataset is archived. The combination part of the chain assembles the six intermediate datasets into a single dataset, performing cross-calibration between satellites, then derives rates of surface elevation change, and finally derives key performance indicators before product output. The performance indicators are the product accuracy, stability, geographical coverage and percentage of datapoints within a given uncertainty. There is also a validation chain, run yearly, comparing our surface elevation change rates to those from Operation IceBridge (see Studinger 2014). This is described more fully in the related Product Quality Assurance Document (PQAD) [D2].
Figure 1: High-level flow chart of the Antarctic surface elevation rate processing chains. Left: Single mission processing, middle: combined mission processing, right: validation.
All input and auxiliary datasets and models necessary are freely available, although for some registration is required. For more details, including web addresses, see the ‘Input and Auxiliary Data’ section 1.2 of the related Algorithm Theoretical Basis Document (ATBD) [D1].
It should be noted that all the necessary data from historic missions has already been assembled on the University of Leeds system. CryoSat-2 and Sentinel-3A/B data is downloaded to the system on a regular basis.
1.1.2. Surface elevation change, Greenland – D3.IS.6.2
In contrast to the Antarctica surface elevation change processor, the Greenland surface elevation processor system consists of one ingestion chain. However, the amount of data needed for processing depends on the satellite mission. While 5 years of data are used for older satellites (ERS-1, ERS-2 and Envisat), ongoing missions (CryoSat-2, and Sentinel-3) use only 3 years of data. The input datasets used for CDRv3 are ERS1 Phase C Reaper L2, ERS1 Phase G Reaper L2, ERS2 Reaper L2, Envisat GDR v3, CryoSat-2 L2i baseline D, Sentinel-3A L2 and Sentinel-3B L2 (same input data as for D3.IS.6.1). A flowchart of the processing chains is shown in Figure 2.
As data is only ingested upon need, the ERS1, ERS2 and Envisat ingestion only has to be run once. CryoSat-2 and Sentinel-3 data, on the other hand, will be ingested monthly to produce the newest iCDRs. The processing of ERS1, ERS2 and Envisat relies on the optimal combination of the plane-fitting (PF)1 and repeat-track (RT) algorithms. The drifting orbit of CryoSat-2 does not allow for RT solutions; thus, the combined CryoSat-2 and Sentinel-3 (as well as CryoSat-2 alone) surface elevation change is derived only by the PF-algorithm. The processing chain performs mission cross-calibration as a part of the implemented processing algorithm. For more details, see section 2.3 of the related Algorithm Theoretical Basis Document (ATBD) [D1].
The ingestion chain for CryoSat-2 and Sentinel-3A/B will run monthly as long as CryoSat-2 and/or Sentinel-3 is operational. All intermediate datasets are archived. The final processing chain merges these files into a single dataset using kriging to average intermediate monthly elevation rates closest in time to the final-solution timestamp. Finally, key performance indicators are derived before product output. The performance indicators are the product accuracy, stability, geographical coverage and percentage of data-points within a given uncertainty. Following these performance indicators, the validation chain is run annually and compares the derived surface elevation change rates to those from Operation IceBridge (see Studinger 2014). This procedure is described in more detail in the related Product Quality Assurance Document (PQAD) [D2].
All ESA radar altimetry data is downloaded to the storage servers at the Technical university of Denmark on a daily basis. The data-storage forms the basis of the operational system developed within ESA's Climate Change Initiative and is also the backbone of the process in the Ice Sheets and Ice Shelves service of the C3S Land Hydrology and Cryosphere project. All input and auxiliary datasets and models necessary are freely available, although for some registration is required. For more details, including web addresses, see the related Algorithm Theoretical Basis Document [D1], Section 2.2.
Figure 2: High-level flow chart of the Greenland surface elevation rate processing chains. Left: Older mission processing, middle: the flow chart for the newer satellites and the combined mission processing, right: validation.
1.2. Hardware, supercomputers and cloud computing
1.2.1. Surface elevation change, Antarctica – D3.IS.6.1
The team has access to a wide range of computing resources at the University of Leeds comprising a mixture of centrally run University Facilities and more specialist Faculty systems supporting around 200Tb of storage. The University has a High-Performance Computing system with a configuration of around 2000 cores based on Intel Nehalem processors.
The Centre for Polar Observation and Monitoring (CPOM) at the University of Leeds has established a presence on the Joint Analysis System Meeting Infrastructure Needs (JASMIN) facility, which provides infrastructure for environmental data analysis across the United Kingdom. It is operated by the Centre for Environmental Data Analysis (CEDA), funded by the UK's Natural Environment Research Council (NERC). This resource is available to the C3S team.
1.2.2. Surface elevation change, Greenland – D3.IS.6.2
The team at DTU Space have access to a scalable range of computing facilities. A High-Performance cluster with 48 treads has been chosen for the Greenland SEC processing. All development has been done on a macOS system with 4 cores and limited amount of memory and storage capabilities.
2. Upgrade cycle implementation procedure
2.1. Surface elevation change, Antarctica – D3.IS.6.1
CryoSat-2 and Sentinel-3A/B level 2 data is usually available for download approximately 35 days after it is acquired. The system at the University of Leeds checks for new data once per day and downloads whatever is available. Crossover processing requires whole cycles (or pseudo-cycles) of data, which in the case of CryoSat-2 are 30 days long and for Sentinel-3A/B are 27 days long. The processing chain is then started. From data ingestion to the product being made available, it takes approximately 3 days to run. In practice the time lag between data acquisition and product update is about 2 months.
The validation dataset is updated irregularly, but usually yearly, and the validation chain run once it becomes available. The validation chain only takes a few hours to run.
2.2. Surface elevation change, Greenland – D3.IS.6.2
Both Sentinel-3 and CryoSat-2 level-2 data are usually available for download approximately 35 days after they are acquired by the satellite. The system setup at the Technical University of Denmark downloads all new data daily, and the Greenland surface elevation change processors are run every month by an automated processing procedure. When the automated processing is done, a processing summary is sent to all relevant persons at DTU space. When the processing e-mail is received, the final product undergoes human inspection before being pushed to an ftp-site, where the product is released for the CDS. In practice, this will result in a time lag between data acquisition and product update of about 2 months. The validation dataset is updated irregularly, but usually yearly as a result of spring field season, and the validation chain run once it becomes available. The validation chain only takes a few hours to run.
3. Procedures for reprocessing CDR's
3.1. Surface elevation change, Antarctica – D3.IS.6.1
Reprocessing has been done to coincide with yearly improvements to the processing system. The v3 CDR contains all of the ERS1, ERS2 and Sentinel-3A data from the v2 CDR. The Envisat dataset has been upgraded from that used in v2, GDRv2.1, to GDR v3.0. The Envisat single-mission chain was run to update the full-mission dataset, and the v3 combined mission chain incorporated it. Similarly, the CryoSat-2 v2 dataset, baseline C, was upgraded to baseline D. The full CryoSat-2 mission to date was reprocessed and incorporated. Monthly updates to CryoSat-2 and Sentinel-3A/B data are incorporated into v3 iCDRs.
The validation dataset, IceBridge ATM L4 Surface Elevation Rate of Change V001 from https://icebridge.gsfc.nasa.gov, is also updated yearly, but irregularly, with flight campaigns in the Arctic and Antarctic spring. The final flights were performed in November 2019. Validation is performed on a yearly basis to accommodate this schedule, using the full dataset from the project start. This necessitates the re-running of the validation chain from the beginning.
3.2. Surface elevation change, Greenland – D3.IS.6.2
See the preceding Antarctic section (section 3.1), as both Greenland and Antarctic SEC use the same input data and have the same processing schedule.
4. System maintenance and system failures
4.1. Surface elevation change, Antarctica – D3.IS.6.1
The University of Leeds system is fully backed up once per week, with incremental backups on the other 6 days. Backups are held at Leeds and also replicated off-site, and a monthly backup on physical media is also held off-site. System maintenance is kept to short periods and advanced warning is given, so processing can be planned around known outages.
Further, the code needed to run the full processing chain is written in C, IDL and shell scripts, all of which can be implemented on many different replacement systems in the unlikely event of a major failure.
The product will be archived at the University of Leeds, but copies will be pushed to the Earth Observation data Centre (EODC), who will supply them to the Climate Data Store (CDS). Users can only obtain data via the CDS. Thus, failure of the University of Leeds system will not affect the availability of any product already made. In unlikely circumstances failure could delay production of further ICDRs, in which case the CDS will be informed.
4.2. Surface elevation change, Greenland – D3.IS.6.2
The system at DTU Space undergoes an incremental backup each day and additionally, full system backup is performed quarterly, both on physical media. System maintenance is kept to short periods and advanced warning is given. Such system maintenance cycles will not affect the users of the product as the monthly product update cycle can be planned around known system outages. If needed the processing chain can be run on alternative servers, as the code needed to run the full processing chain is written in python 3.5, allowing the system to run on any replacement systems, which is found suitable in the unlikely event of a major failure. The product will be archived at the Technical University of Denmark, but copies will be pushed to the Climate Data Store (CDS). Users can only obtain data via the CDS. Thus, failure of the Technical University of Denmark system will not affect the availability of any product already made. In unlikely circumstances failure could delay production of further ICDRs, in which case the CDS will be informed.
5. User support
5.1. Surface elevation change, Antarctica – D3.IS.6.1
The AIS SEC team has a team account with the Copernicus User Support Jira Service Desk System, to provide level 2 user support, i.e. to answer enquiries specific to the product, by direct interaction with the user through the Jira helpdesk. The team contact list is held by the University of Leeds.
E-mail Address: cpom@leeds.ac.uk
5.2. Surface elevation change, Greenland – D3.IS.6.2
The GrIS SEC and GMB processing team at DTU Space has an account with the Copernicus User Support Jira Service Desk System, to handle level 2 user enquiries through the Jira helpdesk. The user support is provided by:
Contact Person Name: Sebastian Simonsen
E-mail Address: c3s-support@space.dtu.dk
References
Ivins, E. R. and James, T. S., (2005). Antarctic glacial isostatic adjustment: a new assessment. Antarctic Science, 17(4), 541-553
Padman, L., et al (2002). A new tide model for the Antarctic ice shelves and seas. Annals of Glaciology, 34, 247-254
Slater, T., et al, (2018). A new digital elevation model of Antarctica derived from CryoSat-2 altimetry, The Cryosphere, 12, 1551-1562, doi: 10.5194/tc-12-1551-2018
Studinger, M., 2014, updated 2018. IceBridge ATM L4 Surface elevation rate of change, version 1. Boulder, Colorado, USA. NASA Snow and Ice Data Center Distributed Active Archive Centre. DOI: 10.5067/BCW6CI3TXOCY
Zwally, H. Jay, Mario B. Giovinetto, Matthew A. Beckley, and Jack L. Saba, 2012, Antarctic and Greenland Drainage Systems, GSFC Cryospheric Sciences Laboratory at http://icesat4.gsfc.nasa.gov/cryo_data/ant_grn_drainage_systems.php
This document has been produced in the context of the Copernicus Climate Change Service (C3S).
The activities leading to these results have been contracted by the European Centre for Medium-Range Weather Forecasts, operator of C3S on behalf of the European Union (Delegation agreement signed on 11/11/2014). All information in this document is provided "as is" and no guarantee or warranty is given that the information is fit for any particular purpose.
The users thereof use the information at their sole risk and liability. For the avoidance of all doubt , the European Commission and the European Centre for Medium - Range Weather Forecasts have no liability in respect of this document, which is merely representing the author's view.
