Introduction

The TIGGE and S2S datasets are migrated to the new ECMWF Data Store (ECDS) based on the Common Data Store Engine (CDS-E).

  • The current access method  WEB-API  will be changed to the analogical CDS-API.
  • For more details about the migration schedule, visit Decommissioning of ECMWF Public Datasets Service
  • Purpose of this page is to describe the current and future access tools to both datasets.

The current TIGGE and S2S users can easily adapt their retrieval requests following the new CDS-API, MARS like, syntax (see in the table below with the examples). 

  • this kind of requests is generated automatically using dedicated MARS catalogues

All users are encouraged to try the new, ECDS dedicated, version of the retrieval requests, which some may find more user friendly (see new CDS-API ("beautified") example).

  • this kind of requests is generated automatically using forms in ECDS portal
Overview of current and future tools

currentnew
data retrieval
  • WEB-API  (python)
  • direct MARS access**
  • CDS-API (python)
  • direct MARS access**
data discovery

dedicated web portals *

1) dedicated ECDS portal

2) dedicated MARS catalogues

data historythe same as before
main web portalthe same as before

* these tools will be discontinued 

**direct access to MARS is not publicly available

Data access

For data discovery (understanding which data is available) and generation of sample retrieval codes, the dedicated data discovery tools are available (see table above).

For programmatic bigger data retrievals, the python API or direct access to MARS should be used (the direct access to MARS is not publicly available).

Data discovery

Users have two options to browse the available S2S or TIGGE data - new dedicated ECDS portal and new dedicated MARS catalogues.

See exact links in table above with the overview of all available tools.

Data retrievals

Until now, there have been 2 options how to access programmatically S2S or TIGGE data:

The required data specification has been analogical in both options (using MARS syntax).

In the new ECDS, those options are similar:

The CDS-API requests can be using:

  • MARS like syntax
  • CDS-API native syntax
    • which should be more self-explaining ("beautified") for users

Be aware of the difference how date ranges are specified in the current WEB-API and CDS-API MARS like syntax:

  • WEB-API: 2024-02-01/to/2024-02-03
  • CDS-API: 2024-02-01/2024-02-03

Registration

As the ECDS portal shares similar infrastructure as the Copernicus Climate Data Store, the existing user tokens (CDS-API keys) can be reused in ECDS. After first log in, a user must accept the ECDS terms and conditions and TIGGE/S2S licence to get the access to the data.

Be aware that currently to access different data stores (cds.ecmwf.int, ecds.ecmwf.int, etc.) the user must have separate .cdsapirc files with the correct URL for given data store (this can change in near future).

Alternatively the URL and user token can be part of the Python client definition as per Examples below.

Users are asked to pay attention to state correctly details about Affiliation, Thematic activities and Activity sectors during the registration as that information helps to understand users needs and interests, which can trigger future changes of TIGGE or S2S datasets.

Examples of the current and future retrieval requests

exampleMARS languagecurrent  WEB-API new CDS-API (MARS like)new CDS-API ("beautified" - generated by ECDS) *
request
retrieve,
class=ti,
date=2024-02-01/to/2024-02-03,
expver=prod,
grid=0.5/0.5,
levtype=sfc,
origin=ecmf,
param=121/122,
step=6/to/24/by/6,
time=00:00:00,
type=cf,
target="output"
#!/usr/bin/env python
from ecmwfapi import ECMWFDataServer
server = ECMWFDataServer()

server.retrieve({
    "dataset": "tigge",
    "date": "2024-02-01/to/2024-02-03",
    "expver": "prod",
    "grid": "0.5/0.5",
    "levtype": "sfc",
    "origin": "ecmf",
    "param": "121/122",
    "step": "6/to/24/by/6",
    "time": "00:00:00",
    "type": "cf",
    "target": "output"
})
#!/usr/bin/env python
import cdsapi
client = cdsapi.Client()
            
dataset = "tigge-forecasts"

request = {
    "class": "ti",
    "date": "2024-02-01/2024-02-03",
    "expver": "prod",
    "grid": "0.5/0.5",
    "levtype": "sfc",
    "origin": "ecmf",
    "param": "121/122",
    "step": "6/to/24/by/6",
    "time": "00:00:00",
    "type": "cf"
}


target = "output"
client.retrieve(dataset, request, target)
#!/usr/bin/env python
import cdsapi
client = cdsapi.Client()

dataset = "tigge-forecasts"

request = {
    "year": ["2024"],
    "month": ["02"],
    "day": ["01", "02", "03"],
    "grid": "0.5/0.5",
    "level_type": "single_level",
    "origin": "ecmf",
    "variable": ["maximum_2m_temperature_last_6_hours",
       "minimum_2m_temperature_last_6_hours"],
    "leadtime_hour": "6/to/24/by/6",
    "time": "00:00:00",
    "forecast_type": "control_forecast",
    "data_format": "grib"
}

target = "output"
client.retrieve(dataset, request, target)
API key

Direct access to MARS database needed

(not publicly available)


--> cat ~/.ecmwfapirc
{
    "url"   : "https://api.ecmwf.int/v1",
    "key"   : "<personal token>",
    "email" : "<personal email>"
}

--> cat $HOME/.cdsapirc
url: https://ecds.ecmwf.int/api
key: <personal token>

--> cat $HOME/.cdsapirc
url: https://ecds.ecmwf.int/api
key: <personal token>
Alternatively the url and token can be part of the Python client definition:
client = cdsapi.Client(url="https://ecds.ecmwf.int/api", 
key="<personal token>")
Alternatively the url and token can be part of the Python client definition:
client = cdsapi.Client(url="https://ecds.ecmwf.int/api", 
key="<personal token>")

(*) The example of dictionary used to translate MARS like to CDS beautified requests for TIGGE dataset is ecds-tigge.yaml


  • No labels