...
The ECMWF Data Store Service (DSS) refers to the family of common architecture Data Stores that serve data for the Copernicus Climate Change Service (C3S; via the CDS), Copernicus Atmosphere Monitoring Service (CAMS, via the ADS) and Copernicus Emergency Management Service (CEMS; via the EWDS) (refer to ECMWF data ECDS?).
Based on user and developer experience of the Climate Data Store (CDS) Toolbox, the CDS-Toolbox has been replaced by a JupyterHub service for the online computing environment and earthkit for the quality assured software. Jupyterhub sessions will be available to all DSS users and provide fast access to data available on the various Data Stores and will allow users to perform post-processing and visualisation of this data. The sessions are considered small (see provision details below) and not designed for very large computation. For larger computation task, users should consider other JupyterHub resources, for example WEkEO.
| Note | ||
|---|---|---|
| ||
All JupyterLab JupyterHub sessions running on this service are time limited. When the time is up, the instance will be killed automatically along with any active processing that may be taking place. You can only have one JupyterLab instance running. If you left one running, JupyterHub will connect you straight back into it. |
...
The DSS JupyterHub will be available from the ECMWF JupyterHub launcher page, linked from the Data Store web-sites (maybe not ADS/EWDS, maybe other data-stores too, update when decided). Access requires ECMWF log in credentials (as required by the Data Store web-site to download data), including a two-factor authentication.
...
DSS users will be able to spawn sessions with one of the environments summarised in the table below. This can be selected from the "Select an Environment" dropdown selector on the JupyterHub Launcher, and depending on the environment, additional options may be configurable. Please note that this list may be expanded/reduced as the service evolves to meet the needs of users.
Name | Use case | RAM | CPU | Duration |
|---|---|---|---|---|
ECMWF Data Store Service - SMALL | Downloading, inspecting and plotting data products found on the Data-Stores Please note that session type is too small for installing additional python packages | 2 | 1 | 12 hours |
ECMWF Data Store Service - MEDIUM | Some small data processing, e.g. data averaging of small files | 4 | 2 | 3 hours |
...
| Info | ||
|---|---|---|
| ||
In the first deployment smaller (and shorter) Small sessions will be prioritised to ensure fair usage of the platform. These priorities are to be monitored closely and will evolve as the project develops. |
Pre-installed software
Python environment
The default python environment is created using conda-forge with a python version versoin 3.11.10 and the following environment.yml file:
...