This tutorial is meant to support the training activity on the web API that may occur in the ECMWF's classroom.
The API is developed in pure python and the latest version is made available via Python Package Index (PyPI)
Installing the web API
The API installation within the ECMWF premises can- On an classroom account
- On a mac/Linux notebook
- On a notebook with windows installed
Disabling the proxy settings
Retrieving data
Installing the web API on a classroom account
Enabling a virtual environment
Users do not have root access. In order to allow them to install python packages they have to setup a virtual environment.
Those with root access, or that have already configured a working python environment can go tho the next session
To fully cover the topic of installing virtual environments, users can look at this tutorial Using virtual Python environment.
Users that want to permanently and quickly access the virtual environment have to create a file $HOME/.bashrc and put the following content in it.
Code Block | ||
---|---|---|
| ||
# Use a sensible umask umask 022 # Place to keep your virtual environment export WORKON_HOME="$HOME/.virtualenvs" mkdir -p $WORKON_HOME # Use Python interpreter from /usr/local/apps python_home="/usr/local/apps/python/2.7.12-01" # Use our Python interpreter for virtualenvwrapper export VIRTUALENVWRAPPER_PYTHON="$python_home/bin/python" export VIRTUALENVWRAPPER_VIRTUALENV="$python_home/bin/virtualenv" export VIRTUALENVWRAPPER_VIRTUALENV_ARGS="--no-site-packages" . $python_home/bin/virtualenvwrapper.sh |
Users that do not want to have the virtual environment to be available at each access they can use a different file e.g. use_environments.
The python version defined in the python_home environmental variable may change with time (new versions available).
To check which version are available: type module avail python and press enter.
To be able to use one specific version, just type module load [your choiche] and then check if the virtualenv command is available.
With the python ready to use and the script with the configurations to execute the virtualenv command, we have to source the script to configure our shell and then install a virtual environment.
Code Block | ||
---|---|---|
| ||
source ~/.bashrc # (or source use_environments) mkvirtualenv cdsapi # or add -p [path to python executable you wish to use it can be different from the one specified in the set up script] workon cdsapi # to activate the environment / deactivate # to deactivate the environment |
After entering the virtual environment using the command workon, we can use pip to install any package available via PyPI.
Installing the API using pip
Just execute:
pip install cdsapi
Or in case of root install sudo pip install cdsapi
For non linux/mac users
The point is to download and install your favorite python version then install the command pip, following common instructions, set up the command line environment to point to python and pip and then run pip install cdsapi.
Using the API
Installing credentials
Users have to locate theyr credentials on the climate data store.
In order to have credentials, users must register in the climate data store.
Registered users can access https://cds.climate.copernicus.eu/api-how-to and locate the fieeld named "Install the CDS API key" .
The two lines specifying url and key must be copied ina specific file: $HOME/.cdsapirc
Non linux like users at the moment can use the credentials within the applications without using the file .cdsapirc (this means that they will produce code exposing the credentials which is a bad habit).
Disabling the ECMWF's proxy settings
For users using non ECMWF's computers (windows and othere notebooks) this step is not necessary.
In order to use the web api we need that in the environment running it the proxy settings are deactivated (this because the CDS is within the premesis)
To achieve this goal, in the console with the cdsapi environment type:
Code Block | ||
---|---|---|
| ||
export http_proxy= export https_proxy= |
Just type enter after = !
Retrieving data
The 2 examples presented in https://cds.climate.copernicus.eu/api-how-to are a good and simple examples working under unix like systems
Code Block | ||
---|---|---|
| ||
#!/usr/bin/env python import cdsapi c = cdsapi.Client() c.retrieve("reanalysis-era5-pressure-levels", { "variable": "temperature", "pressure_level": "1000", "product_type": "reanalysis", "date": "2008-01-01", "time": "12:00", "format": "grib" }, "download.grib") |
Code Block | ||
---|---|---|
| ||
#!/usr/bin/env python import cdsapi c = cdsapi.Client() c.retrieve("insitu-glaciers-elevation-mass", { "variable": "elevation_change", "format": "tgz" }, "download.tar.gz") |
In case the user is using python under windows, the script need to be changed.
The change involve onlu the code line c = cdsapi.Client()
It must be changed in:
Background Color | ||
---|---|---|
| ||
c = cdsapi.Client(api_key=[your key as specified in the api-how-to], end_point=''https://cds.climate.copernicus.eu/api/v2") |
More hands on example:
Code Block | ||
---|---|---|
| ||
#!/usr/bin/env python import cdsapi from datetime import datetime, timedelta p_levels = [str(z) for z in ([1] + list(range(50, 1050, 50)))] c = cdsapi.Client() def days_of_month(y, m): d0 = datetime(y, m, 1) d1 = datetime(y, m + 1, 1) out = list() while d0 < d1: out.append(d0.strftime('%Y-%m-%d')) d0 += timedelta(days=1) return out for y in range(2008, 2018): for m in range(1,13): for d in days_of_month(y, m): c.retrieve("reanalysis-era5-pressure-levels", { "variable": "temperature", "pressure_level": p_levels, "product_type": "reanalysis", "date": d, "time":[ '00:00','01:00','02:00', '03:00','04:00','05:00', '06:00','07:00','08:00', '09:00','10:00','11:00', '12:00','13:00','14:00', '15:00','16:00','17:00', '18:00','19:00','20:00', '21:00','22:00','23:00' ], "format": "netcdf" }, "ea_t_{day}.nc".format(day=d) ) |