Versions Compared

Old Version 12

changes.mady.by.user Karen O'Regan

Saved on

compared with

New Version Current

changes.mady.by.user Karen O'Regan

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Table of Contents


Note
titleQuick Guide
  • Create a new EWDS-Beta account and accept EWDS-Beta Terms & Conditions at https://ewds-beta.climate.copernicus.eu/
  • Accept the licences of the product you are interested in (go to the Download form of the product of interest).
  • For API access:
    • Update API credentials and end point (eg. in your .cdsapirc file) as shown, after logging on here (Note that new .cdsapirc file should only contain a URL and a key field - there is NO UID field anymore).
    • Upgrade your CDS API (i.e pip install --upgrade cdsapi). Please make sure you are using a supported version of Python i.e. 3.8 and above. 
    • New API is designed to be backward compatible which means that once your .cdsapirc is updated and your CDS API package upgraded, your API script may run successfully immediately, downloading data from CDS-Beta. There are a few datasets however for which API syntax may have changed - please check the warning on the catalogue page of the affected datasets (e.g. E-OBS). 
    • Run a small API test request. 

More details available (e.g. CDS Toolbox and Good-to-know sections below)


Introduction 

A new state-of-the-art infrastructure, referred to as the Common Data Store (CDS) Engine, has been developed by ECMWF to host the Climate and Atmospheric Data Stores.  All layers of the infrastructure are being modernised: the front-end web interface, the back-end software engine, and the underlying cloud infrastructure hosting the service and core data repositories.

...

As a consequence of the single sign-on implementation, your current CDS or ADS login credentials will not work on the new CDS, ADS, EWDS infrastructure, regardless of whether you are currently using the CDS or ADS interactively (e.g. over the web interface) or programmatically (using the API service). A new UserID (ECMWF) and API key will be needed.

Warning

Your ECMWF account will only allow you to login to the new CDS/ADS/EWDS, not onto the current CDS/ADS.

You cannot use your ECMWF account to log in onto the current CDS or current ADS. SSO does not apply to current CDS nor current ADS.

Your existing CDS account will only work in the current CDS, not in EWDS Beta. You need an ECMWF account to start using EWDS-Beta.

CDS/ADS/EWDS Catalogue - data download over web interface: 

  • Before downloading any data from the new CDS/ADS/EWDS infrastructure, users will have to accept dataset terms and conditions licences on the new system. Users' profiles including licensing information are not being transferred. In practical terms, this means for example that if you are downloading ERA5 data (web or API) from the current CDS, you will need to login to the new CDS and accept the terms and conditions for ERA5 (i.e. Licence to use Copernicus Products). You can check whether you have accepted the dataset licence under Your Profile on CDS-Beta (see Dataset Licences section below the Terms & Conditions)

API Service - programmatic way of downloading data: 

  • A new version of the cdsapi package has been released. API users need to upgrade their CDS API (i.e pip install --upgrade cdsapi). Note that the new version of the cdsapi package works on both the current and new CDS. 
  • When EW- DS Beta is open, log Log in to the EW- DS Beta interface using your ECMWF account and follow the instructions online. Make sure you also accept the Terms and Conditions of the datasets you wish to download (this is the dataset licence in addition to the Terms & Conditions for using EWDS Beta - see CDS/ADS/EWDS Catalogue section above). 
  • Before submitting your API request, you will You will need to update your .cdsapirc file as shown in the first black box displayed on the EWDS Beta API page (you must be logged in!). Note that new .cdsapirc file should only contain a URL and a key field - there is NO UID field anymore)
  • While the new API is designed to be backward compatible, we strongly recommend you Before running your API script to fetch data from EWDS Beta, please use the "Show API request code" tool on the dataset Download Form on EWDS Beta to check you are using the correct syntax for your API request. CDS/ADS/EWDS API syntax is changed and some keys or parameter names may have also changed .  for some datasets (e.g. E-OBS).


CDS Toolbox, Viewers and Applications:

...

  • New publication of datasets (including publication of existing dataset new versions) will only be made in the new CDS/ADS/EWDS infrastructure. Users are therefore urged to migrate over to the new CDS/ADS/EWDS infrastructure as soon as possible.
  • Some observation datasets will not be immediately available to download from the new CDS infrastructure while technical issues with these particular datasets are being addressed.
  • Some unsupported datasets in current CDS will be deprecated in the new CDS infrastructure. /ADS/EWDS infrastructure.
  • A new GRIB to NetCDF convertor has been implemented in CDS/EWDS-Beta. Further details to be provided soon.
  • For the time being, DOIs of datasets in CDS-Beta will point to the same datasets in the current CDS until the current CDS is shut down in September 2024.
  • The current CDS and ADS will be completely shut down in September 2024. Please migrate to using the new CDS/ADS/EWDS by September 3rd16th, 2024. Should users still have API scripts pointing to the retired CDS and ADS systems, these will systematically break and no support will be provided.
  • For more information on specific changes to the CDS and ADS systems, visit: Informative page in readiness for CDS-Beta

...