GRIB files
The instructions in this article show ways to convert GRIB to netCDF for use in other analysis and visualisation software.GRIB files are the standard format used at ECMWF. Files retrieved from the MARS archive, or from the public data server for reanalysis products such as ERA-Interim are all GRIB format. GRIB is a WMO standard.
The OpenIFS/IFS models output fields in GRIB format. These files are a mix of GRIB-1 & GRIB-2 format messages, the multi-level fields are encoded as GRIB-2, whereas surface fields are GRIB-1. GRIB files can also contain multiple vertical coordinates: pressure levels, model levels, sub-surface levels etc. This can cause a problem with some 3rd party tools, as the same GRIB variable code is used for each axis. The instructions below show how to work around this by splitting the file to separate fields on different vertical axes.
Considering the output from OpenIFS, the ICMGG files contain all the gridpoint fields. The ICMSH files are the spherical harmonics of the wind fields, pressure and temperature. See controlling the OpenIFS output for more details of output fields and options.
Grib-api tools
Please note that grib-api is being phased out in preference for ecCodes. All the grib_* commands shown below are still available with ecCodes.
grib_to_netcdf
This command will convert one or more GRIB files to netCDF and is available with the ECMWF ecCodes/grib-api software (for grib-api, versions above 1.11.0 are recommended).
For more details please see: grib_to_netcdf description.
grib_to_netcdf only works correctly when a single level coordinate is present in the GRIB file. Often the model output files have fields on multiple level types (ie. hybrid model levels and pressure levels).
grib_copy ICMGGftkm+001440 ICMGG_[typeOfLevel].grb
In this example, the GRIB model output file ICMGGftkm+001440 contains a number of different model level types. The special square bracket "[ ]" syntax is recognised by grib_copy (and other grib commands such as grib_filter) and can contain any valid GRIB key.
This example will copy the original file, separating the level types into their own file: ICMGG_hybrid.grb, ICMGG_isobaricInhPA.grb, ICMGG_surface ...and so on.
grib_to_netcdf can then be used on the individual files:
grib_to_netcdf ICMGG_hybrid.grb -o ICMGG_hybrid.nc
To convert from vorticity and divergence to wind u and v, please see CDO instructions below.
Note that grib_to_netcdf does not perform any regridding. If the fields use a reduced Gaussian latitude grid, they will not be converted to a regular grid. Use CDO instead to do this.
Please see 'Using Metview with OpenIFS' for more details.
NCAR command language (NCL)
NCL provides a tool to convert both GRIB-1 & GRIB-2 to netCDF called ncl_convert2nc.
NCL example scripts to convert to netCDF are also available.
CDO: Climate Data Operators
These instructions assume the use of a recent version of Climate Data Operators (CDO) (available from Max-Planck-Institut, Germany).
CDO supports GRIB-2 but needs either the GRIB_API or ecCodes library from ECMWF to be included. This is necessary in order to work correctly with OpenIFS model output.
cdo --version
Climate Data Operators version 1.7.2 (http://mpimet.mpg.de/cdo)
...........
CDI library version : 1.7.2 of Aug 10 2016 09:22:26
CGRIBEX library version : 1.7.5 of Jun 3 2016 14:44:00
GRIB_API library version : 1.15.0
NetCDF library version : 4.4.1 of Aug 3 2016 11:10:49 $
.........
If you find the CDO commands below do not work, you can either build CDO yourself, making sure a recent version of grib_api or ecCodes is used, or you can use the workaround below.
Interpolate from reduced to regular grid
CDO can be used to interpolate from the reduced Gaussian grid to a regular Gaussian grid (IFS/OpenIFS resolutions T21 & T42 use a regular Gaussian grid)
cdo -R copy <input grib> <output grib>
If the -R option does not work, use 'cdo setgridtype,regular' instead.
Steps to convert GRIB to netCDF
Split z axis
Before converting to netCDF, separate the different vertical axes using the generic command: cdo splitzaxis <input file> <output file pattern>.
This will ensure the following steps work correctly.
cdo splitzaxis ICMSHg4a4+000000.grb ICMSHg4a4+000000_split ls ICMSH*split* ICMSHg4a4+000000_split01.grb ICMSHg4a4+000000_split02.grb ICMSHg4a4+000000_split03.grb ICMSHg4a4+000000_split04.grb
In this example, the different files might contain: '01' - fields on pressure levels, '02' - fields on model levels, '03' - hybrid levels, '04' - surface and so on. The number of files created depends on the number of different types of levels. Use the command 'grib_ls' to inspect the contents of each split file.
Note that the filename ICMSH in this example, indicates it contains spectral and not gridpoint fields.
Another way to split the file would be using the grib_copy command from the grib-api (or ecCodes) library as shown above:
grib_copy ICMSHg4a4+000000.grb ICMSHg4a4+000000_[typeOfLevel].grb
In this case 'typeOfLevel' is a GRIB key. The square brackets is a special syntax to the grib_copy command. This approach works with any GRIB key.
Convert spectral to gridpoint
CDO supports two options for converting spectral to gridpoint data:
- sp2gp - converts spectral to a quadratic Gaussian grid.
- sp2gpl - converts spectral to a linear Gaussian grid, appropriate for IFS data.
Both produce a regular Gaussian grid. For more details please see the description of these operators on the CDO homepage.
Convert the resulting output files to netcdf.
cdo -f nc copy <input grib> <output netcdf>
If you prefer wind components u & v instead of vorticity and divergence, use the cdo command:
cdo dv2uvl <input file> <output file>
To interpolate to pressure levels from model levels, use this cdo command, for example:
cdo ml2pl,92500,85000,50000,20000 <input file> <output file>
Viewing contents of GRIB files
There are various commands for inspecting the contents of a GRIB file. The GRIB_API installation as part of OpenIFS has useful commands like grib_ls and grib_dump.
grib_ls ICMSHfrq2+000000.grb grib_dump ICMSHfrq2+000000.grb
Here is another example using the cdo command:
$ cdo sinfo ICMSHfrq2+000000.grb
File format: GRIB
-1 : Institut Source Param Time Typ Grid Size Num Levels Num
1 : ECMWF unknown 11.3 var P16 65792 1 60 1
2 : ECMWF unknown 39.3 var P16 65792 1 60 1
3 : ECMWF unknown 43.3 var P16 65792 1 60 1
4 : ECMWF unknown 44.3 var P16 65792 1 60 1
5 : ECMWF unknown 152.128 var P16 65792 1 1 2
6 : ECMWF unknown 6.3 var P16 65792 1 1 2
Horizontal grids :
1 : spectral > size : dim = 65792 truncation = 255 spc = 32896
complexPacking = 1
Vertical grids :
1 : hybrid level : 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35
36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51
52 53 54 55 56 57 58 59 60
2 : hybrid level : 1
Time axis : 1 step
RefTime = 1999-12-24 12:00:00 Units = hours Calendar = PROLEPTIC
YYYY-MM-DD hh:mm:ss YYYY-MM-DD hh:mm:ss YYYY-MM-DD hh:mm:ss YYYY-MM-DD hh:mm:ss
1999-12-24 12:00:00
cdo sinfo: Processed 6 variables over 1 timestep. ( 0.04s )
Examples
Converting a single file
In this example a single GRIB file with model level data, on a regular Gaussian grid is converted to netCDF with vorticity & divergence replaced by u & v and model level data interpolated to pressure levels.
cdo -f nc copy file.grb file.nc cdo dv2uvl file.nc file_uv.nc cdo sp2gpl file_uv.nc file_uv_gg.nc cdo ml2pl,92500,85000,50000,20000 file_uv_gg.nc file_uv_gg_pl.nc
Converting series of files
The following script shows how to loop over several output files using the steps above to convert to netCDF. Note the spectral parameter files are also converted but the extra command using the sp2gpl operator is first used to convert the variables from spectral to gridpoint on a regular, linear, Gaussian grid.
In this example, we use the pipelining feature of CDO to do both the 'copy' and 'sp2gpl' steps in one command, keeping the entire operations in memory for efficiency.
for i in 000 024 048 072
do
cdo -R copy ICMGGfpp4+000${i}.grb ICMGGfpp4+000${i}_R.grb
cdo splitzaxis ICMGGfpp4+000${i}_R.grb ICMGGfpp4+000${i}
cdo -f nc copy ICMGGfpp4+000${i}01.grb ICMGGfpp4+000${i}_PL_GP.nc # pressure level gridpoint fields
cdo -f nc copy ICMGGfpp4+000${i}02.grb ICMGGfpp4+000${i}_ML_GP.nc # model level gridpoint fields
cdo splitzaxis ICMSHfpp4+000${i}.grb ICMSHfpp4+000${i}
cdo -f nc copy -sp2gpl ICMSHfpp4+000${i}01.grb ICMSHfpp4+000${i}01_PL_GP.grb # spectral fields on press levels transformed to gridpoint
cdo -f nc copy -sp2gpl ICMSHfpp4+000${i}03.grb ICMSHfpp4+000${i}03_ML_GP.grb # spectral fields on model levels transformed to gridpoint
cdo -f nc copy -sp2gpl ICMSHfpp4+000${i}04.grb ICMSHfpp4+000${i}04_surf_GP.grb # spectral fields on surface levels transformed to gridpoint
done
Possible problems and solutions
CDO does not understand GRIB-2 messages
If you are using an old version of CDO, it may not understand GRIB-2 messages. CDO must be compiled with either grib-api or ecCodes library in order to handle GRIB-2.
A workaround is to use the grib_set command (from the grib_api/ecCodes installation) to change the GRIB edition number for all messages.
grib_set -s editionNumber=1 <input grib2 file> <output grib1 file>
Note this will only work if you are using less than 128 model levels as it does not actually convert the message to GRIB-1 if there are any GRIB-2 specific elements in the messages.
Parameter names are lost for GRIB-1 fields
ECMWF GRIB-1 use keys which may not be recognised by CDO because they are locally defined (e.g. the shortName key) and not defined in the WMO GRIB tables that CDO uses. This can cause parameter names to be lost or not recognised when using the CDO commands.
A workaround is to use the -t ecmwf option. The -t option tells CDO to use the predefined ECMWF parameter tables (see CDO documentation for more details).
cdo -t ecmwf -f nc copy mygrib1.grb mygrib1.nc
Do not use this option for GRIB-2 fields. It sets the GRIB table default to be specific to the GRIB-1 ECMWF tables.
Acknowledgements
Thanks to Paul Dando of User Support for help with the contents of this page.