Contributors: T. Usedly (Deutscher Wetterdienst)
Issued by: Deutscher Wetterdienst (DWD) / Tim Usedly
Date: 13/11/2023
Ref: C3S2_D312a_Lot1.3.2.7_202311_SQAD_DWD_CLD_SRB_ERB_v1.1
Official reference number service contract: 2021/C3S2_312a_Lot1_DWD/SC1
History of Modification
List of datasets covered by this document
Related documents
Acronyms
List of figures
General definitions
Table 1: Definition of various technical terms used in the document
| Jargon | Definition |
|---|---|
| Brokered product | The C3S Climate Data Store (CDS) provides both data produced specifically for C3S and so-called brokered products. The latter are existing products produced under an independent programme or project which are made available through the CDS. |
| Climate Data Store | The front-end and delivery mechanism for data made available through C3S. |
| TCDR | It is a consistently-processed time series of a geophysical variable of sufficient length and quality. |
| ICDR | An Interim Climate Data Record (ICDR) denotes an extension of TCDR, processed with a processing system as consistent as possible to the generation of TCDR. |
| CDR | A Climate Data Record (CDR) is defined as a time series of measurements with sufficient length, consistency, and continuity to determine climate variability and change. |
| CM SAF Web User Interface (WUI) | Part of the EUMETSAT CM SAF website providing the service to download the products via an user-friendly interface. |
Scope of the document
This document describes the systems used for delivering the Thematic Climate Data Record (TCDR) and Interim Climate Data Record (ICDR) on Cloud Properties, Surface Radiation Budget and Earth Radiation Budget, brokered from EUMETSAT's CM SAF to the Climate Data Store (CDS) of the Copernicus Climate Change Service (C3S). Interfaces to external data and the interface to the CDS are outlined. Information about new data cycles is also provided in this document. Internal procedures for managing and communicating system maintenance and for user support are described.
Executive summary
The TCDR and ICDR on the Essential Climate Variables (ECVs) Cloud Properties, Surface Radiation Budget and Earth Radiation Budget are a brokered product of CLARA-A3 dataset and its ICDR continuation produced by EUMETSAT's CM SAF. Within C3S, the datasets are made accessible to the CDS via the webserver hosted by DWD (https://c3s.satproj.klima.dwd.de/data/) and can be ordered via HTTP/HTTPS protocol. Login details are provided directly to the CDS team. Respective outages will be kept to a minimum and, whenever possible, communicated to C3S in advance. We provide user support for queries relating to our products when forwarded to us by C3S.
1. System Overview
1.1 System elements and interfaces
The CLARA-A3 TCDR and ICDR have been produced and archived outside of this contract by EUMETSAT's CM SAF. Therefore, the interfaces for the delivering of data to C3S are described within this document. Detailed Information on the retrieval algorithm can be found in the CM SAF Algorithm Theoretical Basis Document (ATBD) [D1], details of the product can be found in the Product User Manual (PUM) [D2].
1.1.1 Interfaces to external data
1.1.1.1 Brokered CDR products
The products of the CLARA-A3 TCDR and ICDR dataset on Cloud Properties, Surface Radiation and Earth Radiation Budget were downloaded via the CM SAF Web User Interface (https://wui.cmsaf.eu/safira/action/viewProduktSearch) and copied to the webserver hosted by DWD (see Section 1.1.2).
1.1.2 Delivery of the data to C3S
The brokered CLARA-A3 data products are made accessible via DWD's webserver at https://c3s.satproj.klima.dwd.de/data/. All data provided for the CDS by DWD will be uploaded to this server. It can be accessed via HTTP/HTTPS protocol. Exact addresses of the provided files will be provided to the CDS upon submission. This allows automated access to the files, e.g. via scripts handling the delivery chain of an order in the CDS from the server to the user. We aim for a 99% availability of the data server, with 95% of the data transactions happening at a rate of 10 Mbit/s or faster.
1.2 Hardware, supercomputers and cloud computing
As mentioned previously, the CLARA-A3 dataset on Cloud Properties, Surface Radiation Budget and Earth Radiation Budget has been produced outside of this contract. The following describes the computing infrastructure used for storing and delivering the data to C3S.
1.2.1 Back-up
The Cloud Properties, Surface Radiadtion Budget and Earth Radiation Budget dataset brokered from EUMETSAT's CM SAF is not backed up within the framework of C3S, but in the case of data corruption or loss on the data server, which serves as interface to the CDS, it can be downloaded again from its original, dedicated source, where back-ups outside the C3S project responsibilities are maintained.
2. Upgrade cycle implementation procedure
The first brokered edition of the CLARA-A3 Cloud Properties, Surface Radiadtion Budget and Earth Radiation Budget dataset (TCDR v3.0) covering the years 01/1979 - 12/2020 was delivered in 06/2023. At the same time the first of three ICDR's (within the current project phase) was delivered covering the time period 01/2021 - 10/2022. The remaining two ICDR deliveries will be brokered to the CDS in the same way and are scheduled for 12/2023 (covering 11/2022 - 04/2023) and 06/2024 (05/2023 - 10/2023).
3. Procedures for reprocessing CDR's
Within the scope of this project, no reprocessing of the CLARA-A3 TCDR and ICDR on Cloud Properties, Surface Radiation Budget and Earth Radiation Budget is foreseen. Therefore n/a
4. System maintenance and system failures
In general, any issues with IT infrastructure that either delays delivery of data or impedes access to data via the CDS will be documented in the Quarterly Report.
4.1 Interfaces to external data
This section discusses treatment of delays at interfaces to external data sources.
4.1.1 Brokered TCDRs and ICDRs
It is beyond the scope of our responsibility to safeguard the stability of services and data servers of external providers. If we become aware of issues at their end that will impede the timely delivery of brokered data, we will notify C3S.
4.2 Interface to the CDS
This section discusses maintenance and outages of the data server hosting the files for access through the CDS.
4.2.1 Planned maintenance
Planned outages of the data server on which the CDS accesses the files are kept to a minimum. However, data transition and software updates may become necessary, or hardware may need to be maintained or replaced. Such planned outages will be communicated to the CDS as early as possible. Respective notifications will be issued by DWD's IT department and delivered to the DWD-based members of the Service Management Team of C3S2_312a_Lot1, who will ensure prompt passing on to the CDS. We do not expect to contact users of datasets submitted by us directly, but only through this communication to CDS.
We are open to request by C3S to schedule dates on which no planned maintenance involving an outage should be carried out. Although a respective commitment by the IT department cannot be guaranteed, we always strive to respect such requests.
4.2.2 Unplanned outages
Unexpected system outages will be treated according to internal regulations and procedures at DWD's IT department. This involves notification of the DWD-based members of the Service Management Team, who will in turn notify C3S of the outage as quickly as possible. The IT department will resolve the issue according to their procedures and notify C3S, again via the Service Management Team.
5. User Support
5.1 Enquiries
The user has the possibility to browse and search in the Copernicus Knowledge Base (CKB) or the Copernicus Climate Data Store desk. This is known as the level-0 process of User Support (Figure 5-1).
Figure 5-1: The schematic of Copernicus User Support (CUS Handbook, section 2)
Contact and User Support process on JIRA Service Desk
Once a request is sent, the Copernicus User Support (CUS) Service team at ECMWF will handle the request within 8 hours (level-1).
For any scientific and special enquiries that cannot be answered by the CUS team at ECMWF or addressed to the Knowledge Base, the request will be forwarded to the Copernicus User Support Specialists (level-2).
Enquiries forwarded to the Copernicus User Support Specialist team will be acknowledged within 3 working days (target 100%) and a notification sent to the user. In case of specific scientific issues, the enquiries will be channelled to the ECV and data specialist of the C3S2_312a_Lot1 project and should be resolved within 3 working weeks (target 85%). In each quarter, we aim for User Support satisfaction scoring 3 in 90% of all voluntary based feedbacks by users, with 1 (very unsatisfied) to 5 (very satisfied). We will also list the number of tickets in the Quarterly Report.
Related articles
-
ERB TSI: Product User Guide and Specification (PUGS) (Copernicus Knowledge Base)
-
ERB TSI: Product Quality Assurance Document (PQAD) (Copernicus Knowledge Base)
-
ERB TSI: Product Quality Assessment Report (PQAR) (Copernicus Knowledge Base)
