Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

 

Section
Column
Model output

GRIB files

Excerpt

The instructions in this article show how to convert GRIB to netCDF for use in other analysis and visualisation software.

GRIB files are the standard format used at ECMWF. Files retrieved from the MARS archive, or from the public data server for reanalysis products such as ERA-Interim are all GRIB format.GRIB is a WMO standard.

The OpenIFS/IFS models output fields The OpenIFS model outputs files in GRIB format. These files are a mix of GRIB 1 & GRIB 2 format messages. They , the multi-level fields are encoded as GRIB-2, whereas surface fields are GRIB-1. GRIB files can also contain multiple vertical coordinates: pressure levels, model levels, sub-surface levels etc, as well as fields on reduced Gaussian grids. This can cause a problem with some 3rd party tools as the same GRIB variable code is used for each axis. The instructions below show how to work around this.

The Considering the output from OpenIFS, the ICMGG files contain all the gridpoint fields. The ICMSH files are the spherical harmonics of the wind fields, pressure and temperature. See controlling the OpenIFS output for more details of output fields and options.

Grib-api tools

Please note that grib-api is being phased out in preference for ecCodes. All the grib_* commands shown below are still available with ecCodes.

grib_to_netcdf

This command will convert one or more GRIB files to netCDF and is available with the ECMWF ecCodes/grib_-api software (versions above 1.11.0 are recommended. It is not available in versions less than 1.10.0).

For more details please see: grib_to_netcdf description.

grib_to_netcdf only works correctly when a single level coordinate is present in the GRIB file. Often the model output files have fields on multiple level types (ie. hybrid model levels and pressure levels).

Code Block
titleSplit GRIB file into separate level types
grib_copy ICMGGftkm+001440 ICMGG_[typeOfLevel].grb

In this example, the GRIB model output file ICMGGftkm+001440 contains a number of different model level types. The special square bracket "[ ]" syntax is recognised by grib_copy (and other grib commands such as grib_filter) and can contain any valid GRIB key.

This example will copy the original file, separating the level types into their own file: ICMGG_hybrid.grb, ICMGG_isobaricInhPA.grb, ICMGG_surface ...etc.The same square bracket syntax is also understood by the grib_filter commandand so on.

grib_to_netcdf can then be used on the individual files:

Code Block
titleConvert GRIB file to netCDF
grib_to_netcdf ICMGG_hybrid.grb -o ICMGG_hybrid.nc

To convert from vorticity and divergence to wind u and v, please see CDO instructions below.

Note that grib_to_netcdf does not perform any regridding. If the fields use a reduced Gaussian latitude grid, they will not be converted to a regular grid. Use CDO instead to do this.

Column
width320px
Panel
bgColorwhite
titleBGColorlightlightgrey
titleOn this page

Table of Contents
indent10px

Info
The MetView application is available for analysis and visualisation of OpenIFS output.
Please see 'Using Metview with OpenIFS' for more details.

 

NCAR command language (NCL)

NCL provides a tool to convert both GRIB1 & GRIB2 to netCDF called ncl_convert2nc.

NCL example scripts to convert to netCDF are also available.

CDO

...

: Climate Data Operators

These instructions assume the use of a recent version of Climate Data Operators (CDO) (available from Max-Planck-Institut, Germany).

CDO supports GRIB2 but needs the GRIB_API library from ECMWF in order to work correctly with OpenIFS model output.

Code Block
languagebash
titleCheck CDO is compiled with grib-api (or ecCodes)
cdo --version
Climate Data Operators version 1.7.2 (http://mpimet.mpg.de/cdo)
...........
     CDI library version : 1.7.2 of Aug 10 2016 09:22:26
 CGRIBEX library version : 1.7.5 of Jun  3 2016 14:44:00
GRIB_API library version : 1.15.0
  NetCDF library version : 4.4.1 of Aug  3 2016 11:10:49 $
.........

If you find the CDO commands below do not work, you can either build CDO yourself, making sure a recent version of grib_api or ecCodes is used, or you can use a workaround below. Use the grib_set command (from the grib_api installation) to change the GRIB edition number for all messages.

Code Block
titleWorkaround if CDO does not understand GRIB2
grib_set -s editionNumber=1 <input grib2 file> <output grib1 file>
Note

Note this will only work if you are using less than 128 model levels as it does not actually convert the message to GRIB1 if there are any GRIB2 specific elements in the messages.

Interpolate from reduced to regular grid

CDO CDO can be used to interpolate from the reduced Gaussian grid to a regular Gaussian grid (model IFS/OpenIFS resolutions T21 & T42 use a regular Gaussian grid)

...

Info

If the -R option does not work, use 'cdo setgridtype,regular' instead.

...

Steps to convert GRIB to netCDF

Split z axis

Before converting to netCDF, separate the different vertical axes using the generic command: cdo splitzaxis <input file> <output file pattern>.

This will ensure the following commands work correctly.

Code Block
titleExample: Separate multiple z axes into individual files
cdo splitzaxis <input file> <output file>

...

ICMSHg4a4+000000.grb ICMSHg4a4+000000_split
ls ICMSH*split*
ICMSHg4a4+000000_split01.grb
ICMSHg4a4+000000_split02.grb
ICMSHg4a4+000000_split03.grb
ICMSHg4a4+000000_split04.grb

In this example, the different files might contain '01' fields on pressure levels, '02' fields on model levels, '03' hybrid levels, '04' surface and so on. The number of files created depends on the number of different types of levels. Use the command 'grib_ls' to inspect the contents of each split file.

Note that the filename 'ICMSH' in this example, indicates it contains spectral and not gridpoint fields.

Another way to split the file would be using the grib_copy command from the grib-api (or ecCodes) library:

Code Block
grib_copy ICMSHg4a4+000000.grb ICMSHg4a4+000000_[typeOfLevel].grb

In this case 'typeOfLevel' is a GRIB key. The square brackets is a special syntax to the  grib_copy command. This approach works with any GRIB key.

Convert spectral to gridpoint

CDO supports two options for converting spectral to gridpoint data:

  • sp2gp    - converts spectral to a quadratic Gaussian grid.
  • sp2gpl   - converts spectral to a linear Gaussian grid, appropriate for IFS data.

Both produce a regular Gaussian grid. For more details please see the description of these operators on the CDO homepage.

Convert the resulting output files to netcdf.

Code Block
titleConvert GRIB file to netCDF file
cdo -t ecmwf -f nc copy <input grib> <output netcdf>

...

Code Block
titleConvert vorticity & divergence to u & v
cdo -t ecmwf dv2uvl <input file> <output file>

...

Code Block
titleInterpolate model levels to pressure levels
cdo -t ecmwf ml2pl,92500,85000,50000,20000 <input file> <output file>

...

In this example a single GRIB file with model level data, on a regular Gaussian grid is converted to netCDF with vorticity & divergence replaced by u & v and model level data interpolated to pressure levels.

Code Block
grib_set -s editionNumber=1 file file.grb
cdo -t ecmwf -f nc copy file.grb file.nc
cdo -t ecmwf dv2uvl file.nc file_uv.nc
cdo -t ecmwf sp2gpl file_uv.nc file_uv_gg.nc
cdo -t ecmwf ml2pl,92500,85000,50000,20000 file_uv_gg.nc file_uv_gg_pl.nc

...

The following script shows how to loop over several output files using the steps above to convert to netCDF. Note the spherical harmonic spectral parameter files are also converted but the extra command using the sp2gpl operator is first used to convert the variables from spherical harmonics spectral to gridpoint.

Code Block
titleSample script to convert OpenIFS GRIB output to netCDF
for i in 000 024 048 072
do

grib_set -s editionNumber=1 ICMGGfpp4+000${i} ICMGGfpp4+000${i}.grb
cdo -R copy ICMGGfpp4+000${i}.grb ICMGGfpp4+000${i}_R.grb
cdo splitzaxis ICMGGfpp4+000${i}_R.grb ICMGGfpp4+000${i}
cdo -t ecmwf -f nc copy ICMGGfpp4+000${i}01.grb ICMGGfpp4+000${i}_PLGG.nc
cdo -t ecmwf -f nc copy ICMGGfpp4+000${i}02.grb ICMGGfpp4+000${i}_MLGG.nc

grib_set -s editionNumber=1 ICMSHfpp4+000${i} ICMSHfpp4+000${i}.grb
cdo splitzaxis ICMSHfpp4+000${i}.grb ICMSHfpp4+000${i}
cdo sp2gpl ICMSHfpp4+000${i}01.grb ICMSHfpp4+000${i}01_GP.grb
cdo sp2gpl ICMSHfpp4+000${i}03.grb ICMSHfpp4+000${i}03_GP.grb
cdo sp2gpl ICMSHfpp4+000${i}04.grb ICMSHfpp4+000${i}04_GP.grb
cdo -t ecmwf -f nc copy ICMSHfpp4+000${i}03_GP.grb ICMSHfpp4+000${i}_ML.nc
cdo -t ecmwf -f nc copy ICMSHfpp4+000${i}01_GP.grb ICMSHfpp4+000${i}_PL.nc
cdo -t ecmwf -f nc copy ICMSHfpp4+000${i}04_GP.grb ICMSHfpp4+000${i}_surf.nc

done

 

Possible problems and solutions

 

Workaround if CDO does not understand GRIB2 messages

If you are using an old version of CDO, or one that does not use the grib-api or ecCodes libraries, Use the grib_set command (from the grib_api installation) to change the GRIB edition number for all messages.

Code Block
titleWorkaround if CDO does not understand GRIB2
grib_set -s editionNumber=1 <input grib2 file> <output grib1 file>

 

 

 

Note

Note this will only work if you are using less than 128 model levels as it does not actually convert the message to GRIB1 if there are any GRIB2 specific elements in the messages.

 

Parameter names are lost

The -t option tells CDO to use the predefined ECMWF parameter tables (see CDO documentation for more details). Without this option parameter names are not preserved. If the - t option doesn't work this is usually because CDO is using an old version of the ECMWF GRIB_API software.

 

 

HTML
<script type="text/javascript" src="https://software.ecmwf.int/issues/s/en_UKet2vtj/787/12/1.2.5/_/download/batch/com.atlassian.jira.collector.plugin.jira-issue-collector-plugin:issuecollector/com.atlassian.jira.collector.plugin.jira-issue-collector-plugin:issuecollector.js?collectorId=5fd84ec6"></script>

Excerpt Include
Credits
Credits
nopaneltrue