Open CEMS-Flood Data

CEMS-Flood data comes primarily in GRIB2 format.

To read GRIB files we encourage using Python 'xarray' and 'cfgrib' packages.

This guideline provides instructions about how to install required libraries (assuming you are working on a Linux OS) and documents the datasets' specific configurations that must be set when reading GRIBs.

Tools and libraries installation

First of all install Conda, a Python packages and environments manager.

Then open a terminal and type:

Set up the python local environment and install required packages

# create a local virtual environment, you can call it as you wish, here 'myenv' is used.
conda create -n myenv python=3.8 

# add repository channel
conda config --add channels conda-forge

# activate the local environment. 
conda activate myenv

# install the required packages
conda install -c conda-forge/label/main xarray cfgrib eccodes

# make sure you have installed eccodes version >= 2.23.0 
python -c "import eccodes; print(eccodes.__version__)"

Start a python console (it is important that you have activated the local environment) and type:

Open a GRIB file

In [1]: import xarray as xr

In [2]: ds = xr.open_dataset('download.grib',engine='cfgrib')

In [3]: ds
Out[4]:
<xarray.Dataset>
Dimensions:     (latitude: 1500, longitude: 3600, step: 3, time: 3)
Coordinates:
    number      int64 ...
  * time        (time) datetime64[ns] 2019-12-01 2019-12-02 2019-12-03
  * step        (step) timedelta64[ns] 1 days 2 days 3 days
    surface     int64 ...
  * latitude    (latitude) float64 89.95 89.85 89.75 ... -59.75 -59.85 -59.95
  * longitude   (longitude) float64 -179.9 -179.8 -179.8 ... 179.7 179.8 179.9
    valid_time  (time, step) datetime64[ns] ...
Data variables:
    dis24       (time, step, latitude, longitude) float32 ...
Attributes:
    GRIB_edition:            2
    GRIB_centre:             ecmf
    GRIB_centreDescription:  European Centre for Medium-Range Weather Forecasts
    GRIB_subCentre:          0
    Conventions:             CF-1.7
    institution:             European Centre for Medium-Range Weather Forecasts
    history:                 2021-02-11T11:00:21 GRIB to CDM+CF via cfgrib-0....

Dataset's specific cfgrib configurations

The different GRIB data structures of the EFAS and GloFAS datasets may require some additional configuration to be set in the backend_kwargs argument of the xarray.open_dataset function.

Read GRIB historical datasets:

CEMS-Floods offers two historical datasets: GloFAS and EFAS historical.

Read historical

import xarray as xr

ds = xr.open_dataset("glofas_historical_201901.grib",engine="cfgrib",backend_kwargs={'time_dims':['time']})

Read GRIB GloFAS historical datasets with multiple product types:

GloFAS historical has two product types, consolidated and intermediate, that you can download together in a GRIB file.

In order to open the file you need to specify the experimentVersionNumber in the backend_kwargs:

consolidated: '0001'

intermediate: '0005'

Read GLoFAS historical with multi product types

import xarray as xr
ds = xr.open_dataset('glofas.grib', engine='cfgrib',
...                      backend_kwargs={'read_keys': {'experimentVersionNumber':'0001'}})

Read GRIB file that has multiple product types:

There are 4 datasets that may have more that one product type in a GRIB file:

EFAS forecast: "control reforecast", "ensemble perturbed reforecast", "high resolution forecast"

EFAS reforecast: "control reforecast", "ensemble perturbed reforecast"

GloFAS historical: "consolidated", "intermediate"

GloFAS forecast: "control reforecast", "ensemble perturbed reforecasts"

GloFAS reforecast: "control reforecast", "ensemble perturbed reforecast"

In order to read them you need to specify which product type you are reading using the backend_kwargs:

Read GRIB file with 2 product types

import xarray as xr

# Reading the Control reforecast (cf) data

glofas_cf = xr.open_dataset("Glofas_forecast.grib",  engine='cfgrib', backend_kwargs={'filter_by_keys': {'dataType': 'cf'}, 'indexpath':''}) 


 # Reading the Ensemble perturbed reforecasts (pf) data

glofas_pf = xr.open_dataset("Glofas_forecast.grib ",  engine='cfgrib', backend_kwargs={'filter_by_keys': {'dataType': 'pf'}, 'indexpath':''})

Page tree