Contributors: O. Bobryshev (DWD), N. Rahpoe (DWD)
Issued by: Oleksandr Bobryshev
Date: 12/02/2021
Ref: C3S_D312b_Lot1.2.2.1-v2.0_201907_SQAD_ECVSurfaceRadiationBudget_v1.0
Official reference number service contract: 2018/C3S_312b_Lot1_DWD/SC1
|
|
|
In the scope of the Copernicus Climate Change Service (C3S), two Surface Radiation Budget datasets are combined into one Climate Data Record (CDR) called "CLARA product family".
The first dataset, EUMETSAT's CM SAF CLARA-A2.1, is the core of this CDR. The second dataset includes extra data products that are not included in the CLARA-A2.1 dataset. The core data products are brokered from CM SAF. The extra net fluxes are not included in the brokered CLARA-A2.1 dataset and are calculated specifically within the C3S project as complimentary data for the convenience of users. Their format is as close as possible to the CLARA-A2.1 and the datasets are meant to be used together. Both datasets are frequently updated with Interim Climate Data Records (ICDRs) or simply extensions, generated using the same software and algorithms to cover more recent periods. The ICDR part for all products covers period from 2019 onwards.
This separation into two datasets is necessary to keep the origin of the data, e.g. licence affiliations: "EUMETSAT's CM SAF" and "C3S", clear for individual products (Table 1)
Table 1: Licence overview of the CLARA product family Surface Radiation Products available through the CDS
Year | CDR Type | CLARA Product Family | |||
SIS | SDL, SOL | SRS | Net Fluxes | ||
| Longwave fluxes | Extra data products | |||
1982 – 2018 | TCDR | CM SAF (CLARA A2.1) | CM SAF (CLARA A2.1) | C3S | C3S |
2019-onwards | ICDR | CM SAF (CLARA A2.1 ICDR) | C3S | C3S | C3S |
The longwave fluxes are not included in the CM SAF ICDR plans for the current phase. To ensure the dataset integrity and continuity, they are calculated within the C3S for the ICDR part. As such, the longwave fluxes change their licence affiliation, namely they are provided within the C3S project for the ICDR part (2019-onwards) and are brokered from EUMETSAT’s CM SAF for the TCDR part (1982 to 2018).
Furthermore the CMSAF CLARA-A2.1 dataset has a temporal coverage of January 1982 to June 2019 (as described in [D2]), but TCDR data are only being brokered to the CDS up to December 2018. Data available from the CDS for January 2019 onwards are brokered/derived from the CLARA A2.1 ICDR1.
In contrast to the original CM SAF CLARA-A2.1 dataset, the brokered service within Copernicus solely provides:
An aggregated version of all satellite data – excluding the provision of the individual satellite datasets.
1 The CMSAF CLARA A2.1 and CLARA-A2.1 ICDR have a 6 month overlap from 2019/01 to 2019/06. One of the input datasets used in CLARA-A2.1, namely ERA-interim reanalysis data, is only available until August 2019 and dates beyond that are only covered by the newer ECMWF reanalysis version (ERA-5) and ECMWF IFS model. The CLARA-A2.1 ICDR uses ERA data from January 2019 onwards. The 6-month overlap between CLARA-A2.1 and CLARA-A2.1 ICDR is needed to ensure a sufficient period for data comparison and validation. CM SAF ATBD-ICDR (not released yet) contains more information on the overlap period. |
This document describes the systems used for delivering two Surface Radiation Budget products to the Climate Data Store (CDS) of the Copernicus Climate Change Service (C3S). The first dataset is the CLARA-A2.1 data record (CLARA-A2.1: CM SAF CLoud, Albedo and Radiation data record – AVHRR based, Edition 2.1) and the second dataset includes extra products, produced specifically within the C3S project, that are not included in the CLARA-A2.1 CDR.
Interfaces to external data, the infrastructure for data processing (if applicable), and the interface to the CDS are outlined. Information about new data cycles and reprocessing is also provided in this document. Internal procedures for managing and communicating system maintenance and for user support are described.
Both datasets are made accessible via DWD's webserver at https://c3s.satproj.klima.dwd.de/data/ and can be ordered via HTTP protocol. The data are password-protected. Login details are provided directly to the CDS team. Respective outages will be kept to a minimum and, whenever possible, communicated to C3S in advance.
We provide user support for queries relating to our products when forwarded to us by C3S (see Chapter 5 User Support).
The products of the CM SAF CLARA dataset on surface radiation were downloaded via the CM SAF Web User Interface and copied to the webserver hosted by DWD (see Section 1.2.3).
The extra data products dataset are not included in the brokered CLARA-A2.1 dataset and are calculated specifically within the C3S project, 312bLot1, as complimentary data for the convenience of users.
The CLARA product family CDR is made accessible via DWD's webserver at https://c3s.satproj.klima.dwd.de/data/ and can be ordered via HTTP protocol. The data are password-protected. Login details are provided directly to the CDS team. This allows automated access to the files, e.g. via scripts handling the delivery chain of an order in the CDS from the server to the user.
We aim for a 99% availability of the data server, with 95% of the data transactions happening at a rate of 10 Mbit/s or faster.
DWD hosts the webserver (CentOs Linux) that can be accessed from outside the DWD firewall via HTTP. Monitoring of server availability is routinely performed using Nagios IT Infrastructure Monitoring Tool. The check is performed every 60 second and if the data on the webserver are unavailable, an automatic email notification is issued.
Preliminary analysis of the required disk space for data sets provided by DWD within C3S_312b_Lot1 shows that the quota on the data server dedicated to DWD's efforts within C3S_312b_Lot1 will not be exceeded.
The CLARA-A2.1 data comes from an external source, where back-ups outside our responsibilities are maintained.
The script to generate extra data products and respective metadata is automated. It is backed up by the IT-department of DWD. Another copy is stored at the ECFS (ECMWF's File Storage System). In the case of data corruption or loss of the data server, which serves as an interface to the CDS, the CLARA data can be downloaded from their source and extra data products regenerated using the automated script. The estimated time needed for full recovery of the data is 3 hours.
The CLARA Product family is periodically updated with Interim Climate Data Records (ICDRs) generated using the same software and algorithms to cover more recent periods. The ICDR part for all products covers period from 2019 onwards. The longwave fluxes are not included in the CM SAF ICDR plans, so to ensure the dataset integrity and continuity, they are calculated within the C3S for the ICDR part. As such, the longwave fluxes change their license affiliation from CM SAF to C3S for the ICDR part.
Not applicable
In general, any issues with IT infrastructure that either delays delivery of data or impedes data access via the CDS will be documented in the Quarterly Reports.
This section discusses treatment of delays at interfaces to external data sources.
It is beyond the scope of our responsibility to safeguard the stability of services and data servers of external providers. If we become aware of issues at their end that will impede the timely delivery of brokered data, we will notify C3S.
The production of extra data products is made on the DWD servers. The script to generate extra data products and respective metadata is automated. It is backed up by the IT-department of DWD. Another copy is stored at the ECFS (ECMWF's File Storage System). In the case of data corruption or loss of the data server, which serves as an interface to the CDS.
This section discusses maintenance and outages of the data server hosting the files for access through the CDS.
Planned outages of the data server on which the CDS accesses the files are kept to a minimum. However, software updates may become necessary, or hardware may need to be maintained or replaced. Such planned outages will be communicated to the CDS as early as possible. Respective notifications will be issued by DWD's IT department and delivered to the DWD-based members of the Service Management Team of C3S_312b_Lot1, who will ensure prompt passing on to the CDS. We do not expect to contact users (of data sets submitted by us) directly, but only through this communication to CDS.
We are open to requests by C3S to schedule dates on which no planned maintenance involving an outage should be carried out. Although a respective commitment by the IT department cannot be guaranteed, we always strive to respect such requests.
Unexpected system outages will be treated according to internal regulations and procedures at DWD's IT department. This involves notification of the DWD-based members of the Service Management Team, who will in turn notify C3S of the outage as quickly as possible. The IT department will resolve the issue according to their procedures and notify C3S, again via the Service Management Team.
The user (customer) has the possibility to browse and search in the Copernicus Knowledge Base (CKB) (CAMS and C3S Knowledge Base (CKB)) or the Copernicus Climate Data Store desk (https://cds.climate.copernicus.eu/cdsapp#!/usersupport). This is known as the level-0 process of User Support.
Figure 1: The schematic of Copernicus User Support (Copernicus User Support Handbook, V2.1)
If the level-0 support fails to answer the user's query, they may then make a request, which is sent to the Copernicus User Support (CUS) Service team at ECMWF. These (level-1) requests will be handled within 8 hours.
For any scientific and specialist enquiries that cannot be answered by the CUS team at ECMWF or addressed by the Copernicus Knowledge Base, the request will be forwarded to the Copernicus User Support Specialists (level-2).
Enquiries forwarded to the Copernicus User Support Specialist team will be acknowledged within 3 working days (target 100%) and a notification sent to the user. This step is handled on the JIRA
JIRA: Project management software tool created by Atlassian company |
Service Desk established at ECMWF for the current project. In case of specific scientific issues, the enquiries will be channeled to the ECV and data specialist of the C3S2_312a_Lot1 project and should be resolved within 3 working weeks (target 85%). In case of brokered datasets, the data specialist may choose to contact the responsible team at the provider level if they cannot resolve the issue on their own. In each yearly quarter, we aim for User Support satisfaction scoring 3 in 90% of all voluntary based feedbacks by users, with 1 (very unsatisfied) to 5 (very satisfied). We will also list the number of tickets raised by users on the JIRA system in the Quarterly Report.
This document has been produced in the context of the Copernicus Climate Change Service (C3S). The activities leading to these results have been contracted by the European Centre for Medium-Range Weather Forecasts, operator of C3S on behalf of the European Union (Delegation agreement signed on 11/11/2014). All information in this document is provided "as is" and no guarantee or warranty is given that the information is fit for any particular purpose. The users thereof use the information at their sole risk and liability. For the avoidance of all doubt , the European Commission and the European Centre for Medium - Range Weather Forecasts have no liability in respect of this document, which is merely representing the author's view. |
Related articles appear here based on the labels you select. Click to edit the macro and add or change labels.