To help users begin planning for the migration, we have produced an initial static sample dataset which provides sample data encoded in GRIB edition 2 from: 

• the analysis and forecast from the control (ex-HRES):
  • atmosphere:  stream=oper, type=and type=fc
  • wave: stream=wave, type=an and type=fc
• the perturbed members from the ENS forecast 
  
  • atmosphere stream=enfo, type=pf
  • wave:  stream=waef, type=pf

Sample data encoded in GRIB2 from the ensemble hindcasts, sub-seasonal and sub-seasonal hindcasts will also be made available in due course.

The sample data are based on the output from the current IFS cy49r1 output for the 00 UTC cycle on 30 November 2024 but encoded in GRIB2 as we currently expect the data which will be produced by IFS cy51r1 to be encoded.  All fields provided are global and interpolated to a 1.0° x1.0° regular latitude-longitude grid.

• Data on constant pressure levels (type=pl) are provided at level=100 only.
  
• Data on theta levels (type=pt) are provided at level=300 for type=fc and level=265 only for type=an.
• Data on potential vorticity levels (levtype=pv) are provided at level=1500 only.
• Parameters accumulated from the start of the forecast have the ecCodes startStep key consistently set to 0 and, consequently, stepRange=0-24 in GRIB2 compared to stepRange=24, etc, in GRIB1
• Parameters accumulated from the start of the forecast are not provided at stepRange=0. 

Structure of static sample dataset on disk

Data are ordered in a MARS-like folder structure under <root_dir>/<class>/<stream>/<type>/<levtype>.  Shown below is a diagrammatic layout of the directory structure under <root_dir>=/ec/vol/mtg2_sample/: 

<root_dir>/od/enfo
└── pf
    ├── pl
    ├── pt
    ├── pv
    ├── sfc
    └── sol

<root_dir>/od/oper
├── an
│   ├── ml
│   ├── pl
│   ├── pt
│   ├── pv
│   ├── sfc
│   └── sol
└── fc
    ├── ml
    ├── pl
    ├── pt
    ├── pv
    ├── sfc
    └── sol

<root_dir>/od/scda
├── an
│   ├── ml
│   ├── pl
│   ├── pt
│   ├── pv
│   ├── sfc
│   └── sol
└── fc
    ├── ml
    ├── pl
    ├── pt
    ├── pv
    ├── sfc
    └── sol

<root_dir>/od/scwv
├── an
│   └── sfc
└── fc
    └── sfc

<root_dir>/od/waef
└── pf
    └── sfc

<root_dir>/od/wave
├── an
│   └── sfc
└── fc
    └── sfc

Structure of filenames

Each directory at the end of the tree contains one file per parameter containing all forecast time steps.  The files are named <paramId>_<class>_<stream>_<type>_<levtype>_<shortName>_<year>_<expver>_<gridType>.grib2 and are based on the ecCodes paramId, shortName and gridType and the MARS class, stream, type, levtype used for the current operational output at IFS cy49r1.

For example, the file named 151_od_oper_fc_sfc_msl_2024_0001_regular_ll.grib2 in the <root_dir>/od/oper/fc/sfc/ directory contains the GRIB2 encoding of mean sea-level pressure from the Control (ex-HRES) forecast at all available forecast steps. 

Who should use the initial static sample dataset ?

The initial static sample dataset is aimed at GRIB encoding and decoding specialists that need early access to samples of ECMWF GRIB2 data to test and prepare any decoding software used that is not based on ECMWF ecCodes GRIB encoding and decoding software to handle the GRIB2 ECMWF data.  The dataset is not suitable for the general users of ECMWF data that want to test their full processing chain.

Sample data for the general user will be made available in the coming months when it will be possible to download the data from the ECMWF MARS archive and process the data using versions of ECMWF software, such as ecCodes and Metview that fully support the new ECMWF GRIB2 encoding.

What can be tested using the static sample dataset ?

The static sample dataset can be used for making some basic tests using your GRIB decoder to ensure the data can be read correctly.  This testing is encouraged if you use a third-party GRIB decoder, such as wgrib2, which is not based on ecCodes.  If your decoder is based on ecCodes then it will need to use a new ecCodes version which will be made available soon.

The static sample dataset can be decoded with the current version of ecCodes but some keys, in particular the shortName and paramId, may not be set correctly.  However, it is possible use the grib_dump tool to inspect the GRIB headers and here the keys are set to the correct values as far as we are aware.

In general, the static sample dataset  is not suitable for initialising an NWP model.  Although encoding examples of all parameters at all forecast steps are available, for those parameters on constant pressure levels, for example, only the 1hPa level is provided.  However, users that initialise NWP models from ECMWF data should note the change in the way the parameters on soil levels are provided as this may require changes to your code.  Instead of having four separate parameters at each soil layer, in GRIB2 there will be one parameter available at each of the four layers (levels).

Changes to parameters

Surface fields - levType=sfc

Soil temperature, volumetric soil moisture and sea-ice temperature

In GRIB1, soil temperature, volumetric soil moisture and sea-ice temperature have different paramIds for each of the 4 soil layers.  In GRIB2, only a single paramId is used with the layer indicated by the level key.

For example, in GRIB1, there is a paramId for Volumetric soil water layer 1 and a different paramId for Volumetric soil water layer 2, etc.  In GRIB2, only a single paramId is used for Volumetric soil water, with the layer 1 having level=1 and layer 2 having level=2. 

These parameters will be archived in MARS with levtype=sol (surface other levels).  To retrieve these parameters in GRIB2, users will need to change their MARS requests to specify levtype=sol,levelist=1/2/3/4.

Parameters statistically processed over a time window

In GRIB1, parameters that are statistically processed over a time window, such and maximum, minimum and average values over the last 3 or 6 hours, have different paramIds for each time window available.  In GRIB2, there is a unique paramId indicating the parameter and type of statistical processing, e.g., paramId=237167, shortName=max_2t for maximum temperature at 2 metres.  A separate key, called statisticalwindow, is then used to specify the time window.  For example, paramId=23167, statisticalwindow=3h identifies maximum temperature at 2 metres during the last 3 hours.

This change applies to:

• Maximum 10 metre wind gust.
• Maximum and minimum temperature at 2 metres.
• Maximum and minimum total precipitation rate.
• Maximum CAPE and CAPE shear.
• Average lightning flash density.
• Most severe and most frequent precipitation type.

To retrieve these parameters in GRIB2 from MARS, users will need to use the new paramId and also specify the statistical processing time window using a new MARS keyword that is to be introduced. 

Maximum 10m wind gust

Maximum and minimum 2m temperature

Maximum and minimum total precipitation rate

Maximum CAPE and CAPE Shear

Mean total lightning flash density

Most severe and most frequent precipitation type

Ocean model output

The paramId's and shortNames for the following output from the ocean model have changed.  In addition:

• the unit of Sea surface practical salinity and Average sea water practical salinity in the upper 300 m has changed from psu to g/kg;
• the unit of the Average sea water potential temperature in the upper 300m has changed from degrees Celsius to Kelvin.

Fluxes of chemical species

In GRIB1, parameters that are fluxes of chemical species have different paramIds for each chemical species and type of flux.  In GRIB2, a unique paramId indicates the type of flux and a new key called chemShortName identifies the chemical.  For example, paramId=435000 refers to the Net ecosystem exchange flux and the chemShortName=CO2 identifies this as the Carbon dioxide net ecosystem exchange flux.

To retrieve these parameters in GRIB2 from MARS, users will need to use the new paramId and also specify the chemical species using the MARS keyword, chem.