Page info | ||||||
---|---|---|---|---|---|---|
|
Info | ||||
---|---|---|---|---|
| ||||
|
Easy Heading Macro |
---|
Introduction
...
Note |
---|
Please be aware that the following page relates to the legacy CDS system. Although some of the functionalities are common to the new CDS system we invite you to read the quick guide and additional information on some of the changes: Please read: CDS and ADS migrating to new infrastructure: Common |
...
IMPORTANT: New CDS API is designed to be backward compatible which means that once your .cdsapirc is updated, your API script may run successfully immediately, downloading data from the new CDS. The content of the page below will be updated in due course to include new CDS information. |
Info | ||||
---|---|---|---|---|
| ||||
|
Easy Heading Macro |
---|
Introduction
The Climate Data Store (CDS) is the cornerstone infrastructure which supports the implementation of the Copernicus Climate Change Service (C3S). It enables the provision of Essential Climate Variables (ECVs), climate analyses, reanalyses, projections and indicators at temporal and spatial scales relevant to adaptation and mitigation strategies for various sectoral and societal benefit areas.
...
Note |
---|
Please make sure that you select all the mandatory fields before submit submitting a request. |
This is shown in the following figures:
...
For non-Python users, please note that the CDS API is REST-based so it can be wrapped by any programming language. Please see this example with the R package: Download CDS ERA5 data using R.
Users can also set the PROXY within the CDS API script:
Code Block | ||
---|---|---|
| ||
import requests import cdsapi session = requests.Session() session.proxies = { 'http': 'http://10.10.10.10:8000', 'https': 'http://10.10.10.10:8000', } client = cdsapi.Client(session=session) |
...
They are divided in three categories: per-user, global and system, and these limits are shown in Table 1 (last reviewed on ).
The CDS will queue requests which would otherwise cause any of these limits to be exceeded. Please note that these limits are changed from time to time according to the current workload of the system and number of concurrent tasks.
...
Expand | ||||||||||||||||||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ||||||||||||||||||||||||||||||||||||||
|
Moreover, CDS data requests also have limits in terms of number of fields and volume size which are different for each dataset (see the tables below). These values are dependent on each dataset's structure and where the data is stored. Again, these restrictions are introduced to help the system maintain good performance and minimise the queue time.
...
Expand | |||||||||||||||||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| |||||||||||||||||||||||||||||||||||||
|
Data hosted on 'MARS external'
Data hosted on 'MARS external'
Data hosted by external contractors
Expand | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Efficiency tips
- Where the data are actually stored can make a significant difference in performance. CDS data hosted in 'MARS internal' is stored on CDS disks, and so is faster to retrieve. The 'MARS external' datasets are stored in the ECMWF MARS (tape) archive, and in this case it is important to request as much data as possible from the same tape file in your CDS request.
- Submit small requests over very large and heavy requests. This will ensure your requests are not penalised in the CDS request queue. In particular:
For ERA5 data requests, please see: How to download ERA5. The following example The first example in CDS API and CDS Toolbox examples section below shows how to efficiently download a whole year of hourly data for 2m temperature (grib format, for an area subset) from the CDS, by asking for one month of data per request:
Expand title Request for hourly data for 2m temperature for 2018 and 2019 Code Block #!/usr/bin/env python3 import cdsapi c = cdsapi.Client() first_year = 2018 last_year = 2019 for year in range(first_year, last_year + 1): for month in range(1, 13): print("=======language py title Request for hourly data for 2m temperature for 2018 and 2019 .
- For C3S seasonal forecast data requests, please see: Recommendations and efficiency tips for C3S seasonal forecast datasets.
- For UERRA data requests, please see: UERRA retrieval efficiency.
- When using the CDS API or the CDS Toolbox, it is advised that users take as an example the API request or toolbox request script shown at the bottom of the CDS web 'Download data' page for the dataset of interest, and use this as the basis for your request.
- Some ERA5 datasets, such as
reanalysis-era5-complete
, do not appear in the CDS catalogue on the web interface, but users can still retrieve the data through the CDS API. In these cases, users can make use of a subset of the ECMWF MARS keywords in their CDS API request, and should also follow the MARS efficiency rule of thumb (the idea is to request as much data as possible from the same tape file or to reduce the number of tapes involved.). - Users can check the allowed and not allowed keywords for the CDS API requests in the following article: Climate Data Store (CDS) API Keywords
CDS API and CDS Toolbox examples
CDS API Request for hourly data of one variable looping though months and years ( e.g 2m temperature for all months in 2018 and 2019)
Code Block | ||||||
---|---|---|---|---|---|---|
| ||||||
#!/usr/bin/env python3 import cdsapi c = cdsapi.Client() first_year = 2018 last_year = 2019 for year in range(first_year, last_year + 1): for month in range(1, 13): print("=========================================================") print("Downloading {year}-{month:02d}".format(year=year, month=month)) c.retrieve( 'reanalysis-era5-single-levels', { 'product_type': 'reanalysis', 'variable': '2m_temperature', 'year': str(year), 'month': "{month:02d}".format(month=month), 'day': [ '01', '02', '03', '04', '05', '06', '07', '08', '09', '10', '11', '12', '13', '14', '15', '16', '17', '18', '19', '20', '21', '22', '23', '24', '25', '26', '27', '28', '29', '30', '31', ], 'time': [ '00:00', '01:00', '02:00', '03:00', '04:00', '05:00', '06:00', '07:00', '08:00', '09:00', '10:00', '11:00', '12:00', '13:00', '14:00', '15:00', '16:00', '17:00', '18:00', '19:00', '20:00', '21:00', '22:00', '23:00', ], 'area': [ 90, 170, 80, 180, ], 'format': 'grib', }, |
...
"{year}-{month:02d}.grib".format(year=year, month=month))
|
CDS API
...
CDS API and CDS Toolbox examples
CDS API Request for Temperature data from the ERA5 hourly dataset on pressure levels ('renalysis-era5-pressure-levels') on 11-10-2018, from 08:00 to 18:00 UTC on pressure level 1000 hPa.
...
Code Block | ||||||||
---|---|---|---|---|---|---|---|---|
| ||||||||
import cdsapi c = cdsapi.Client() c.retrieve('reanalysis-era5-complete', { 'class': 'ea', 'date': '2019-07-01/to/2019-07-31', 'expver': '1', 'levelist': '1/2/3/4/5/6/7/8/9/10/11/12/13/14/15/16/17/18/19/20/21/22/23/24/25/26/27/28/29/30/31/32/33/34/35/36/37/38/39/40/41/42/43/44/45/46/47/48/49/50/51/52/53/54/55/56/57/58/59/60/61/62/63/64/65/66/67/68/69/70/71/72/73/74/75/76/77/78/79/80/81/82/83/84/85/86/87/88/89/90/91/92/93/94/95/96/97/98/99/100/101/102/103/104/105/106/107/108/109/110/111/112/113/114/115/116/117/118/119/120/121/122/123/124/125/126/127/128/129/130/131/132/133/134/135/136/137', 'levtype': 'ml', 'param': '130', 'step': '0', 'stream': 'oper', 'time': '09:00:00', 'type': '4v', }, 'output') |
Info | ||
---|---|---|
| ||
This document has been produced in the context of the Copernicus Climate Change Service (C3S). The activities leading to these results have been contracted by the European Centre for Medium-Range Weather Forecasts, operator of C3S on behalf of the European Union (Delegation agreement Agreement signed on 11/11/2014 and Contribution Agreement signed on 22/07/2021). All information in this document is provided "as is" and no guarantee or warranty is given that the information is fit for any particular purpose. The users thereof use the information at their sole risk and liability. For the avoidance of all doubt , the European Commission and the European Centre for Medium - Range Weather Forecasts have no liability in respect of this document, which is merely representing the author's view. |
Related articles
Content by Label | ||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|