Accessing ECMWF data servers in batch

WARNING: This page has been moved to Access ECMWF Public Datasets.

The ECMWF Public Datasets Service is being decommissioned and access to most datasets closed on 1 June 2023. In a final step later this year, access to the remaining multi-model datasets S2S and TIGGE will be migrated to a different system. For more information and possible alternatives, please consult the dedicated page on Decommissioning of ECMWF Public Datasets service.

Introduction

This service will allow you to access the ECMWF public datasets . You will need to have an account on ECMWF web site. If you don't have an account, please self register here.

The procedure is very simple: the user writes a request, submits it and retrieves a file (grib or netcdf format) including the requested data.

This method is supported on UNIX/Linux and Windows platforms with Python. No assumption should be made regarding service availability and individual user support.

Step-by-step guide

This method allows users to download ECMWF public datasets in a programmatic way.

Install ECMWF key
Install client libraries
Check data availability
Choose dataset
Script examples
Request syntax

Install ECMWF KEY

If you don't have an ECMWF account, please self register at https://api.ecmwf.int/v1/key ( click login and then register ) and then follow ONE of the steps below.

You can also try visiting https://api.ecmwf.int/v1/key/ to retrieve your credentials.

Note that the key expires in 1 year. You will receive an email to the registered email address 1 month before the expiration date with the renewal instructions.

Install client libraries

The client currently supports both Python 2.7.x and Python 3.

The ecmwf-api-client library was added to the Python Package Index (PYPI):

https://pypi.org/project/ecmwf-api-client/

Install via pip with:

pip install ecmwf-api-client

If you do not have admin privileges you can always install the client in your user space:

pip install --user ecmwf-api-client

Alternatively, if you have Anaconda installed you can install the Web API Python client library doing:

conda install -c conda-forge ecmwf-api-client

If you have Anaconda installed you can install the Web API Python client library doing:

conda install -c conda-forge ecmwf-api-client

Alternatively, you can first install Python (and pip) on your Windows environment.

Download windows installer exe from Python.org download page
Run the exe.
Screen will be shown to chose the installation option.
Uncheck "install for all user" option.
Go for the custom installation.
On next screen specify the directory path for which your user have full access on the computer (take a note of this as you will need it to set the environment variables).
Uncheck "create shortcuts for installed application" option.
Make sure "Add python to environment variable" option is Unchecked .
Complete the installation.

Then add the installation and Script folder path in PATH using set (temporary) or setx (permanent) in a Command Prompt window. This is to make sure that your computer knows where to find the Python interpreter. To do this you will have to modify a setting called PATH, which is a list of directories where Windows will look for programs. Further details and examples are available here. You may choose also to set your environment variables into a batch file (e.g. set-env.bat which you will need to run from the Command Prompt.

@echo off

set PY_HOME=C:\...\python37

set PATH=%PY_HOME%;%PY_HOME%\Scripts;%PATH%

You can now install it via pip (see instructions above).

For languages other than Python please see Web-API Downloads

Check Data availability

To check the availability of ECMWF Public Datasets go to the web interface:

http://apps.ecmwf.int/datasets/

Using this interface you can discover all the ECMWF Public Datasets that are available from our archive. We strongly recommend you to to navigate through our public datasets to become familiar with their availability. You may select a Public Dataset and start navigating trough its content.

Please take into account some considerations about the content:

Different ECMWF Public Datasets include different "parameters" , "times", and "steps"
In each ECMWF Public Dataset not all the "parameters" are available from all "steps"
In each ECMWF Public Dataset not all the "steps" are available from all "times"

The web interface above will help you to check and understand the availability. For any kind of selections the system will update the attributes in a dynamic way to reflect the current availability. (i.e. if you change the steps some parameters will be added or removed).

Tip

We encourage users to use the " View MARS request " feature at the bottom of the page once the selection has been done. Using this MARS request you can build your own Python script. If you are interested on the syntax go to Brief request syntax.

If you click on the ERA-interim http://apps.ecmwf.int/datasets/data/interim_full_daily/ you will get a web page reflecting the availability of this specific Public Dataset as the period that it is available, the forecast steps and the parameters that are available etc.
From this page you have the option to define the values of your requests such as the "date", the "time" the "steps" and the "parameters" that you are interested in.
In the case of ERA-Interim if you select time "06:00:00" only, the system updates the availability of the "steps" dynamically. In that case only the "step" 00 is available (checkable).
Additionally if you select "step" 00 you will see that only a subset of the parameters are now available and only these parameters can now be checked.

Spend some time to understand how it works, make some selections try to execute the MARS scripts.

Choosing a dataset

See Available ECMWF Public Datasets.

Script examples

Request syntax

See Brief request syntax

Space shortcuts

Page tree

Related articles

Blog Posts

Recently Updated

8 Comments

Dick Dee

Paul Dando

Matthew Manoussakis

Unknown User (mahg)

Matthew Manoussakis

Dick Dee

Matthew Manoussakis

Arda SORMAN