Contents of the sample static dataset

To help users begin planning for the migration, we have produced an initial static sample dataset which provides sample data encoded in GRIB edition 2 from: 

• the analysis and forecast from the control (ex-HRES):
  • atmosphere:  stream=oper, type=and type=fc
  • wave: stream=wave, type=an and type=fc
• the perturbed members from the ENS forecast 
  
  • atmosphere stream=enfo, type=pf
  • wave:  stream=waef, type=pf

Sample data encoded in GRIB2 from the ensemble post-processed products (ensemble mean, maximum, etc, EFI and Sot), ensemble hindcasts, sub-seasonal and sub-seasonal hindcasts will also be made available in due course.

The sample data are based on the output from the current IFS cy49r1 output for the 00 UTC cycle on 30 November 2024 but encoded in GRIB2 as the data which will be produced by IFS cy51r1 are currently expected to be encoded.  All fields provided are global and interpolated to a 1.0° x1.0° regular latitude-longitude grid.

• Data on constant pressure levels (type=pl) are provided at level=100 only.
  
• Data on theta levels (type=pt) are provided at level=300 for type=fc and level=265 only for type=an.
• Data on potential vorticity levels (levtype=pv) are provided at level=1500 only.
• Data on model levels (levtype=ml) are provided at level=1 only.
• Parameters accumulated from the start of the forecast have the ecCodes startStep key consistently set to 0 and, consequently, stepRange=0-24 in GRIB2 compared to stepRange=24, etc, in GRIB1
• Parameters accumulated from the start of the forecast are not provided at stepRange=0. 

Where to obtain the static sample dataset

• Users with login access to the ECMWF HPC service can access the data from the directory /ec/vol/mtg2_sample. 
• Users without login access to the ECMWF HPC or ECS service can download the data from https://data.ecmwf.int/mtg2_sample.

Directory structure of static sample dataset

Data are ordered in a MARS-like folder structure under <root_dir>/<class>/<stream>/<type>/<levtype>.  Shown below is a diagrammatic layout of the structure under <root_dir>=/ec/vol/mtg2_sample/: 

<root_dir>/od/enfo
└── pf
    ├── pl
    ├── pt
    ├── pv
    ├── sfc
    └── sol

<root_dir>/od/oper
├── an
│   ├── ml
│   ├── pl
│   ├── pt
│   ├── pv
│   ├── sfc
│   └── sol
└── fc
    ├── ml
    ├── pl
    ├── pt
    ├── pv
    ├── sfc
    └── sol

<root_dir>/od/waef
└── pf
    └── sfc

<root_dir>/od/wave
├── an
│   └── sfc
└── fc
    └── sfc

Structure of file names

Each directory at the end of a branch of the tree contains one file per parameter with all forecast time steps.  The files are named <paramId>_<class>_<stream>_<type>_<levtype>_<shortName>_<year>_<expver>_<gridType>.grib2 and are based on the ecCodes paramId, shortName and gridType and the MARS class, stream, type, levtype used for the current operational output at IFS cy49r1.

For example, the file named 151_od_oper_fc_sfc_msl_2024_0001_regular_ll.grib2 in the <root_dir>/od/oper/fc/sfc/ directory contains the GRIB2 encoding of mean sea-level pressure from the Control (ex-HRES) forecast at all available forecast steps. 

Who should use the initial static sample dataset ?

The initial static sample dataset is aimed at GRIB encoding and decoding specialists that need early access to samples of ECMWF GRIB2 data to test and prepare any decoding software used that is not based on ECMWF ecCodes GRIB encoding and decoding software to handle the GRIB2 ECMWF data.  The dataset is not suitable for the general users of ECMWF data that want to test their full processing chain.

Sample data for the general user will be made available in the coming months when it will be possible to download the data from the ECMWF MARS archive and process the data using versions of ECMWF software, such as ecCodes and Metview that fully support the new ECMWF GRIB2 encoding.

What can be tested using the static sample dataset ?

The static sample dataset can be used for making some basic tests using your GRIB decoder to ensure the data can be read correctly.  In particular, testing is encouraged if you use a third-party GRIB decoder, such as wgrib2, which is not based on ecCodes.

If your decoder is based on ecCodes then ecCodes version 2.42.0 should be used  - see ecCodes version 2.42.0 released.  This is available for download from our ecCodes Releases page.

The static sample dataset can be decoded by versions of ecCodes older than 2.42.0 but some keys, in particular the shortName and paramId, may not be set correctly.  However, it is possible use the grib_dump tool to inspect the GRIB headers and here the keys are set to the correct values as far as we are aware.

In general, the static sample dataset  is not suitable for initialising an NWP model.  Although encoding examples of all parameters at all forecast steps are available, for those parameters on constant pressure levels, for example, only the 1hPa level is provided.  However, users that initialise NWP models from ECMWF data should note the change in the way the parameters on soil levels are provided as this may require changes to your code.  Instead of having four separate parameters at each soil layer, in GRIB2 there will be one parameter available at each of the four layers (levels).

Other sources of ECMWF GRIB2 data that can be used for testing

Along with the static sample dataset, users can also use the ECMWF Open Data to test their processing of GRIB2 data.  All of the open data are encoded in GRIB2.  For more information about these data and how to obtain them, see ECMWF open data: real-time forecasts from IFS and AIFS.

Changes to encoding of parameter paramId, shortName, name and units

The encoding of the ecCodes paramId, shortName, name and units have changed for some surface parameters from the analysis and forecast.  Users may need to adapt their workflows in order to process these parameters successfully.  The main changes affecting parameters provided in the static sample dataset are summarised below.  For a list of all changes to parameter encoding, see

Soil temperature, volumetric soil moisture and sea-ice temperature

In GRIB1, soil temperature, volumetric soil moisture and sea-ice temperature have different paramIds for each of the 4 soil layers.  In GRIB2, only a single paramId is used with the layer indicated by the level key.

For example, in GRIB1, there is a paramId for Volumetric soil water layer 1 and a different paramId for Volumetric soil water layer 2, etc.  In GRIB2, only a single paramId is used for Volumetric soil water, with the layer 1 having level=1 and layer 2 having level=2. 

These parameters will be archived in MARS with levtype=sol (surface other levels).  To retrieve these parameters in GRIB2, users will need to change their MARS requests to specify levtype=sol,levelist=1/2/3/4.

Parameters statistically processed over a time window

In GRIB1, parameters that are statistically processed over a time window, such and maximum, minimum and average values over the last 3 or 6 hours, have different paramIds for each time window available.  In GRIB2, there is a unique paramId indicating the parameter and type of statistical processing, e.g., paramId=237167, shortName=max_2t for maximum temperature at 2 metres.  A separate key, called statisticalwindow, is then used to specify the time window.  For example, paramId=23167, statisticalwindow=3h identifies maximum temperature at 2 metres during the last 3 hours.

This change applies to:

• Maximum 10 metre wind gust.
• Maximum and minimum temperature at 2 metres.
• Maximum and minimum total precipitation rate.
• Maximum CAPE and CAPE shear.
• Average lightning flash density.
• Most severe and most frequent precipitation type.

To retrieve these parameters in GRIB2 from MARS, users will need to use the new paramId and also specify the statistical processing time window using a new MARS keyword - timespan - that is to be introduced. 

Maximum 10m wind gust

Maximum and minimum 2m temperature

Maximum and minimum total precipitation rate

Maximum CAPE and CAPE Shear

Mean total lightning flash density

Most severe and most frequent precipitation type

Ocean model output

The paramId's and shortNames for the following output from the ocean model have changed.  In addition:

• the unit of Sea surface practical salinity and Average sea water practical salinity in the upper 300 m has changed from psu to g/kg;
• the unit of the Average sea water potential temperature in the upper 300m has changed from degrees Celsius to Kelvin.

Fluxes of chemical species

In GRIB1, parameters that are fluxes of chemical species have different paramIds for each chemical species and type of flux.  In GRIB2, a unique paramId indicates the type of flux and a new key called chemShortName identifies the chemical.  For example, paramId=435000 refers to the Net ecosystem exchange flux and the chemShortName=CO2 identifies this as the Carbon dioxide net ecosystem exchange flux.

To retrieve these parameters in GRIB2 from MARS, users will need to use the new paramId and also specify the chemical species using the MARS keyword, chem.