Introduction

This page provides technical guidance on accessing and using the ECMWF SOFF dedicated dataset, available to support countries participating in the Systematic Observations Financing Facility (SOFF) initiative.

SOFF, led by the World Meteorological Organization (WMO), aims to strengthen global weather and climate observations by supporting countries—especially those with limited observational capacity—to generate and exchange essential data under the Global Basic Observing Network (GBON). As part of this initiative, ECMWF provides eligible National Meteorological and Hydrological Services (NMHSs) with access to a dedicated set of high-resolution forecast and analysis products in near real time, covering several global regions.

The purpose of this documentation is to help users understand the data structure, formats, and access methods so they can effectively retrieve and use these products in their operational workflows.

How to access the data

The SOFF dedicated dataset is made available through a simple file-based access method. Users can retrieve files in near real time and integrate them into their local workflows.

Access is provided via an FTP service, which allows users to:

• connect using standard FTP clients (graphical or command-line)
• browse directories organised by date and time
• download the required data files

To access the data, users need:

• the FTP server address
• a username (your wmo_xx account) and password (the password is specific for the FTP and can be provided by ECMWF Data Support)
• an internet connection

Once connected, users can navigate the directory structure and retrieve the relevant files based on their needs.

For users with limited technical experience, it is recommended to use a graphical FTP client (such as FileZilla or WinSCP), which provides an easy way to browse and download files without using command-line tools.

Detailed steps and examples are provided below.

FTP url is diss.ecmwf.int

With Linux FTP line command:

> ftp diss.ecmwf.int
# provide username (wmo_xx) and password

# list the available folders
> ls

# move into the folder for a specific day
> cd yyyymmdd

# download all the files in the folder
> mget * .

Filenaming convention and dissemination schedule

Once you access the FTP, you will see different folders in it, corresponding to the last 4 days of data. Folder names follow the format dd-mm-yyyy 00:00.

Each folder contains all the data disseminated for that specific day.

Data is made available according to a defined ECMWF Dissemination schedule, with files added as they become available.

Within each folder, files follow a standard naming convention:

AADMMDDHHIImmddhhiiE where

• AA can be one of the following values:
  • AN (Analysis data)
  • FM (Forecast model level)
  • FP (Forecast pressure level)
  • FS (Forecast surface level)
• D means that the data come from ECMWF HRES data
• MMDDHHII is month, day, hour and minute on which the products are based
• mmddhhii is month, day, hour and minute on which the products are valid at (ii is set to 01 for high resolution forecast time step zero, type=fc, step=0)
• E is the Experiment Version Number’ (as EXPVER in MARS, normally 1).

Example for filename FPD05110000051412001

• FP → Forecast pressure level data

• D → Data from ECMWF IFS Set-I (Ensemble Control)

• 05110000 → Forecast run time

  • 05 = May
  • 11 = day
  • 00:00 = time

• 05141200 → Valid time of the forecast

  • 05 = May
  • 14 = day
  • 12:00 = time

• 1 → Experiment version number

How to read the data - ecCodes Library

To open and read in your GRIB files, ECMWF has developed the ecCodes library, which is ECMWF's primary software package for decoding and encoding GRIB and BUFR messages.

In ecCodes, keys are used to access specific pieces of information (metadata or data) within a GRIB or BUFR message. Each key represents a distinct attribute, such as the parameter name, level, or forecast time.

grib_ls: Quick Summary of GRIB Messages

grib_ls allows users to quickly view key metadata from one or more GRIB messages. 
It prints a table showing key metadata, such as:

• parameter name
• level
• date and time
• forecast step
• etc...

Basic Usage

grib_ls 20250421000000-0h-oper-fc.grib2

Where 20250421000000-0h-oper-fc.grib2 is the name of the file you would like to summarise.

Sample Output

edition      centre       date         dataType     gridType     stepRange    typeOfLevel  level        shortName    packingType
2            ecmf         20250421     fc           regular_ll   0            surface      0            lsm          grid_ccsds

Some useful options also include:

• -p key1,key2,...: Display selected keys only.

• -w key=value: Filter messages by key values.

• -m: Displays the associated MARS keys.

• -n namespace: Displays all the keys belonging to the namespace.

Use Case

You have downloaded a GRIB file containing surface-level forecast data (FSD* file from the FTP) and want to understand what it contains before further processing.

Before using the file for plotting or further processing, you may want to verify:

• which parameters are included (e.g. 2 metre temperature, wind, pressure)
• which forecast steps are available

Solution: use the grib_ls command to get a quick overview of the file:

grib_ls -p shortName,levelType,level,stepRange,validityDate,validityTime 20250504000000-0h-oper-fc.grib2

grib_dump: Detailed Inspection of GRIB Content

grib_dump provides a full and detailed view of the contents of GRIB messages, including both metadata and data values.

It is particularly useful when you need to:

• explore the internal structure of a GRIB file
• understand how data is organised
• debug issues or verify specific values

Unlike grib_ls, which shows a summary, grib_dump outputs all available information, making it a more advanced tool for detailed inspection.

Basic Usage

grib_dump 20250421000000-0h-oper-fc.grib2

Where 20250421000000-0h-oper-fc.grib2 is the name of the file you would like to inspect.

Sample Output (One Message Snippet)

#==============   MESSAGE 160 ( length=723061 )            ==============
GRIB {
  # Meteorological products (grib2/tables/34/0.0.table)
  discipline = 0;
  editionNumber = 2;
  # European Centre for Medium-Range Weather Forecasts (common/c-11.table)
  centre = 98;
  subCentre = 0;
  # Start of forecast (grib2/tables/34/1.2.table)
  significanceOfReferenceTime = 1;
  dataDate = 20250421;
  dataTime = 0;
  # Operational products (grib2/tables/34/1.3.table)
  productionStatusOfProcessedData = 0;
  # Forecast products (grib2/tables/34/1.4.table)
  typeOfProcessedData = 1;
  # MARS labelling (grib2/grib2LocalSectionNumber.98.table)
  grib2LocalSectionNumber = 1;
  # Operational archive (mars/class.table)
  marsClass = 1;
  # Forecast (mars/type.table)
  marsType = 9;
  # Atmospheric model (mars/stream.table)
  marsStream = 1025;
  experimentVersionNumber = 0001;
  numberOfDataPoints = 1038240;
  # There is no appended list (grib2/tables/34/3.11.table)
  interpretationOfNumberOfPoints = 0;
  # Latitude/longitude (Also called equidistant cylindrical, or Plate Carree)  (grib2/tables/34/3.1.table)
  gridDefinitionTemplateNumber = 0;
  # Earth assumed spherical with radius of 6 371 229.0 m (grib2/tables/34/3.2.table)
  shapeOfTheEarth = 6;
  Ni = 1440;
  Nj = 721;
  iScansNegatively = 0;
  jScansPositively = 0;
  jPointsAreConsecutive = 0;
  alternativeRowScanning = 0;
  latitudeOfFirstGridPointInDegrees = 90;
  longitudeOfFirstGridPointInDegrees = 180;
  latitudeOfLastGridPointInDegrees = -90;
  longitudeOfLastGridPointInDegrees = 179.75;
  iDirectionIncrementInDegrees = 0.25;
  jDirectionIncrementInDegrees = 0.25;
  gridType = regular_ll;
  NV = 0;
  # Analysis or forecast at a horizontal level or in a horizontal layer at a point in time (grib2/tables/34/4.0.table)
  productDefinitionTemplateNumber = 0;
  # Temperature (grib2/tables/34/4.1.0.table)
  parameterCategory = 0;
  # Temperature (K)  (grib2/tables/34/4.2.0.0.table)
  parameterNumber = 0;
  #-READ ONLY- parameterUnits = K;
  #-READ ONLY- parameterName = Temperature;
  # Forecast (grib2/tables/34/4.3.table)
  typeOfGeneratingProcess = 2;
  generatingProcessIdentifier = 158;
  # Hour (grib2/tables/34/4.4.table)
  indicatorOfUnitForForecastTime = 1;
  stepUnits = h;
  forecastTime = 0;
  stepRange = 0;
  # Isobaric surface (Pa)  (grib2/tables/34/4.5.table)
  typeOfFirstFixedSurface = 100;
  #-READ ONLY- unitsOfFirstFixedSurface = Pa;
  #-READ ONLY- nameOfFirstFixedSurface = Isobaric surface;
  scaleFactorOfFirstFixedSurface = 0;
  scaledValueOfFirstFixedSurface = 15000;
  # Missing (grib2/tables/34/4.5.table)
  typeOfSecondFixedSurface = 255;
  #-READ ONLY- unitsOfSecondFixedSurface = unknown;
  #-READ ONLY- nameOfSecondFixedSurface = Missing;
  scaleFactorOfSecondFixedSurface = MISSING;
  scaledValueOfSecondFixedSurface = MISSING;
  level = 150;
  shortName = t;
  name = Temperature;
  cfName = air_temperature;
  #-READ ONLY- cfVarName = t;
  modelName = IFS;
  #-READ ONLY- modelVersion = cy49r1;
  numberOfValues = 1038240;
  packingType = grid_ccsds;
  ccsdsFlags = 14;
  ccsdsBlockSize = 32;
  ccsdsRsi = 128;
  # A bit map does not apply to this product (grib2/tables/34/6.0.table)
  bitMapIndicator = 255;
  bitmapPresent = 0;
  values(1038240) =  {
  226.241, 226.241, 226.241, 226.241, 226.241,
  226.241, 226.241, 226.241, 226.241, 226.241,
  226.241, 226.241, 226.241, 226.241, 226.241,
  226.241, 226.241, 226.241, 226.241, 226.241,
  226.241, 226.241, 226.241, 226.241, 226.241,
  226.241, 226.241, 226.241, 226.241, 226.241,
  226.241, 226.241, 226.241, 226.241, 226.241,
  226.241, 226.241, 226.241, 226.241, 226.241,
  226.241, 226.241, 226.241, 226.241, 226.241,
  226.241, 226.241, 226.241, 226.241, 226.241,
  226.241, 226.241, 226.241, 226.241, 226.241,
  226.241, 226.241, 226.241, 226.241, 226.241,
  226.241, 226.241, 226.241, 226.241, 226.241,
  226.241, 226.241, 226.241, 226.241, 226.241,
  226.241, 226.241, 226.241, 226.241, 226.241,
  226.241, 226.241, 226.241, 226.241, 226.241,
  226.241, 226.241, 226.241, 226.241, 226.241,
  226.241, 226.241, 226.241, 226.241, 226.241,
  226.241, 226.241, 226.241, 226.241, 226.241,
  226.241, 226.241, 226.241, 226.241, 226.241
  ... 1038140 more values
  }
  #-READ ONLY- maximum = 230.554;
  #-READ ONLY- minimum = 198.749;
  #-READ ONLY- average = 215.112;
  #-READ ONLY- standardDeviation = 7.31015;
  #-READ ONLY- skewness = 0.149981;
  #-READ ONLY- kurtosis = -1.23212;
  #-READ ONLY- isConstant = 0;
  #-READ ONLY- numberOfMissing = 0;
  #-READ ONLY- getNumberOfValues = 1038240;
}

Some useful options also include:

• -a: If the file contains multiple messages, it will dump all of them (this is the default behaviour, but you can use -a for dumping aliases).

• -p key1,key2,...: If you want to dump specific keys from each message.

• -O: enables Octet mode, producing a WMO documentation-style dump of the GRIB messages.

• -d: prints all data values in the GRIB messages.

grib_to_netcdf: Convert GRIB to NetCDF

ecCodes also allows you to convert your GRIB data into NetCDF files using the grib_to_netcdf command. 

Basic Usage

grib_to_netcdf -o 20250421000000-0h-oper-fc.nc 20250421000000-0h-oper-fc.grib2

Where -o specifies the name of the NetCDF output file and 20250421000000-0h-oper-fc.grib2 is the name you specify for the output file.

Some useful options also include:

• -I key1,key2,...: ignores specific keys during conversion. By default: method, type, stream, refdate, expver are ignored.
• -S key1,key2,...: splits the GRIB file according to specified keys. By default it will split: param, expver.
• -k kind: specifies the type of NetCDF file to create. Possible values are:
  • 1 -> netCDF classic file format
2 -> netCDF 64 bit classic file format (Default)
3 -> netCDF-4 file format
4 -> netCDF-4 classic model file format

grib_copy: Copy Contents of a GRIB File

grib_copy copies the content of GRIB files, printing values of some keys.

Basic Usage

grib_copy 20250421000000-0h-oper-fc.grib2 output.grib2

Where 20250421000000-0h-oper-fc.grib2 is your input file to copy and output.grib2 is the name you specify for the output file.

Note here that using grib_copy without options will take GRIB files with multi-field messages and convert them into single-field messages.

Some useful options also include:

• -w key[:{s|d|i}]{=|!=}value,key[:{s|d|i}]=value,...: a 'Where' clause. Only GRIB messages matching the key/value constraints are copied to the output file. A valid constraint is of type key=value or key!=value. For each key a string (key:s), a double (key:d) or an integer (key:i) type can be defined. Default type is string. In the value, you can also use the forward-slash character '/' to specify an OR condition (i.e. a logical disjunction). Note: only one -w clause is allowed.
• -p key[:{s|d|i}],key[:{s|d|i}],..: declaration of keys to print. For each key a string (key:s), a double (key:d) or an integer (key:i) type can be requested. Default type is string.

How to process the data - some basic recipes

Processing ECMWF model level (ML) data requires additional steps compared to surface or pressure-level data. However, several ready-to-use workflows (“recipes”) are available online.

Examples include:

1. Converting parameters on model levels to GRIB1
   https://medium.com/@valcap74/how-to-run-wrf-model-driven-by-era5-on-model-levels-3c3e31838d22
   https://confluence.ecmwf.int/display/OIFSUF/Grib+to+Netcdf+conversion
2. Using GRIB2 model-level data directly with WRF (with a dedicated Vtable): https://earthscience.stackexchange.com/questions/9497/running-wrf-with-ecmwf-grib2-model-levels

Important note: Since wgrib has separate executables for GRIB1 and GRIB2, wgrib and therefore WPS cannot process mixed files containing GRIB1 and GRIB2.

For this reason:

• Model level (ML) data (GRIB2) and surface (SFC) data (GRIB1) must be stored in separate files
• The ungrib independently for GRIB1 and again for GRIB2, and then, after processing both datasets, you can proceed to nuse metgrid.

Before running ungrib for the second dataset, you must update the relevant settings in the namelist.wps file. 

It is required to make following changes in namelist.wps between two ungrib runs:

&ungrib
 prefix = 'SFILE' 
&ungrib
  prefix = 'MFILE'

To ensure that the two ungrib runs do not overwrite each other.
Then in metgrid section of namelist.wps you need to make sure to use both files created by ungrib runs:

 &metgrid
 fg_name = 'SFILE' 'MFILE' 

What has changed since these recipes are written is that now some GRIB2 parameters are compressed using CCSDS compression which WRF Pre-Processing System cannot decode. Consequently, CCSDS packing in GRIB2 has to be changed to SIMPLE PACKING before any of these steps with:

 grib_set -r -w packingType=grid_ccsds -s packingType=grid_simple ccsds.grib grid_simple.grib