GRIB files
The instructions in this article show how to convert GRIB to netCDF for use in other analysis and visualisation software.GRIB files are the standard format used at ECMWF. Files retrieved from the MARS archive, or from the public data server for reanalysis products such as ERA-Interim are all GRIB format.GRIB is a WMO standard.
The OpenIFS/IFS models output fields in GRIB format. These files are a mix of GRIB 1 & GRIB 2 format messages, the multi-level fields are encoded as GRIB-2, whereas surface fields are GRIB-1. GRIB files can also contain multiple vertical coordinates: pressure levels, model levels, sub-surface levels etc. This can cause a problem with some 3rd party tools as the same GRIB variable code is used for each axis. The instructions below show how to work around this.
Considering the output from OpenIFS, the ICMGG files contain all the gridpoint fields. The ICMSH files are the spherical harmonics of the wind fields, pressure and temperature. See controlling the OpenIFS output for more details of output fields and options.
Grib-api tools
Please note that grib-api is being phased out in preference for ecCodes. All the grib_* commands shown below are still available with ecCodes.
grib_to_netcdf
This command will convert one or more GRIB files to netCDF and is available with the ECMWF ecCodes/grib-api software (versions above 1.11.0 are recommended. It is not available in versions less than 1.10.0).
For more details please see: grib_to_netcdf description.
grib_to_netcdf only works correctly when a single level coordinate is present in the GRIB file. Often the model output files have fields on multiple level types (ie. hybrid model levels and pressure levels).
grib_copy ICMGGftkm+001440 ICMGG_[typeOfLevel].grb
In this example, the GRIB model output file ICMGGftkm+001440 contains a number of different model level types. The special square bracket "[ ]" syntax is recognised by grib_copy (and other grib commands such as grib_filter) and can contain any valid GRIB key.
This example will copy the original file, separating the level types into their own file: ICMGG_hybrid.grb, ICMGG_isobaricInhPA.grb, ICMGG_surface ...and so on.
grib_to_netcdf can then be used on the individual files:
grib_to_netcdf ICMGG_hybrid.grb -o ICMGG_hybrid.nc
To convert from vorticity and divergence to wind u and v, please see CDO instructions below.
Note that grib_to_netcdf does not perform any regridding. If the fields use a reduced Gaussian latitude grid, they will not be converted to a regular grid. Use CDO instead to do this.
Please see 'Using Metview with OpenIFS' for more details.
NCAR command language (NCL)
NCL provides a tool to convert both GRIB1 & GRIB2 to netCDF called ncl_convert2nc.
NCL example scripts to convert to netCDF are also available.
CDO: Climate Data Operators
These instructions assume the use of a recent version of Climate Data Operators (CDO) (available from Max-Planck-Institut, Germany).
CDO supports GRIB2 but needs the GRIB_API library from ECMWF in order to work correctly with OpenIFS model output.
cdo --version Climate Data Operators version 1.7.2 (http://mpimet.mpg.de/cdo) ........... CDI library version : 1.7.2 of Aug 10 2016 09:22:26 CGRIBEX library version : 1.7.5 of Jun 3 2016 14:44:00 GRIB_API library version : 1.15.0 NetCDF library version : 4.4.1 of Aug 3 2016 11:10:49 $ .........
If you find the CDO commands below do not work, you can either build CDO yourself, making sure a recent version of grib_api or ecCodes is used, or you can use a workaround below.
Interpolate from reduced to regular grid
CDO can be used to interpolate from the reduced Gaussian grid to a regular Gaussian grid (IFS/OpenIFS resolutions T21 & T42 use a regular Gaussian grid)
cdo -R copy <input grib> <output grib>
If the -R option does not work, use 'cdo setgridtype,regular
' instead.
Steps to convert GRIB to netCDF
Split z axis
Before converting to netCDF, separate the different vertical axes using the generic command: cdo splitzaxis <input file> <output file pattern>.
This will ensure the following commands work correctly.
cdo splitzaxis ICMSHg4a4+000000.grb ICMSHg4a4+000000_split ls ICMSH*split* ICMSHg4a4+000000_split01.grb ICMSHg4a4+000000_split02.grb ICMSHg4a4+000000_split03.grb ICMSHg4a4+000000_split04.grb
In this example, the different files might contain '01' fields on pressure levels, '02' fields on model levels, '03' hybrid levels, '04' surface and so on. The number of files created depends on the number of different types of levels. Use the command 'grib_ls' to inspect the contents of each split file.
Note that the filename 'ICMSH' in this example, indicates it contains spectral and not gridpoint fields.
Another way to split the file would be using the grib_copy command from the grib-api (or ecCodes) library:
grib_copy ICMSHg4a4+000000.grb ICMSHg4a4+000000_[typeOfLevel].grb
In this case 'typeOfLevel' is a GRIB key. The square brackets is a special syntax to the grib_copy command. This approach works with any GRIB key.
Convert spectral to gridpoint
CDO supports two options for converting spectral to gridpoint data:
- sp2gp - converts spectral to a quadratic Gaussian grid.
- sp2gpl - converts spectral to a linear Gaussian grid, appropriate for IFS data.
Both produce a regular Gaussian grid. For more details please see the description of these operators on the CDO homepage.
Convert the resulting output files to netcdf.
cdo -f nc copy <input grib> <output netcdf>
If you prefer wind components u & v instead of vorticity and divergence, use the cdo command:
cdo dv2uvl <input file> <output file>
To interpolate to pressure levels instead of model levels, use this cdo command for example:
cdo ml2pl,92500,85000,50000,20000 <input file> <output file>
There are various commands for inspecting the contents of a GRIB file. The GRIB_API installation as part of OpenIFS has useful commands like grib_ls
and grib_dump
. Here is another example using the cdo command:
$ cdo sinfo ICMSHfrq2+000000.grb File format: GRIB -1 : Institut Source Param Time Typ Grid Size Num Levels Num 1 : ECMWF unknown 11.3 var P16 65792 1 60 1 2 : ECMWF unknown 39.3 var P16 65792 1 60 1 3 : ECMWF unknown 43.3 var P16 65792 1 60 1 4 : ECMWF unknown 44.3 var P16 65792 1 60 1 5 : ECMWF unknown 152.128 var P16 65792 1 1 2 6 : ECMWF unknown 6.3 var P16 65792 1 1 2 Horizontal grids : 1 : spectral > size : dim = 65792 truncation = 255 spc = 32896 complexPacking = 1 Vertical grids : 1 : hybrid level : 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 2 : hybrid level : 1 Time axis : 1 step RefTime = 1999-12-24 12:00:00 Units = hours Calendar = PROLEPTIC YYYY-MM-DD hh:mm:ss YYYY-MM-DD hh:mm:ss YYYY-MM-DD hh:mm:ss YYYY-MM-DD hh:mm:ss 1999-12-24 12:00:00 cdo sinfo: Processed 6 variables over 1 timestep. ( 0.04s )
Example 1 - Converting a single file
In this example a single GRIB file with model level data, on a regular Gaussian grid is converted to netCDF with vorticity & divergence replaced by u & v and model level data interpolated to pressure levels.
cdo -f nc copy file.grb file.nc cdo dv2uvl file.nc file_uv.nc cdo sp2gpl file_uv.nc file_uv_gg.nc cdo ml2pl,92500,85000,50000,20000 file_uv_gg.nc file_uv_gg_pl.nc
Example 2 - Converting series of files
The following script shows how to loop over several output files using the steps above to convert to netCDF. Note the spectral parameter files are also converted but the extra command using the sp2gpl operator is first used to convert the variables from spectral to gridpoint.
for i in 000 024 048 072 do cdo -R copy ICMGGfpp4+000${i}.grb ICMGGfpp4+000${i}_R.grb cdo splitzaxis ICMGGfpp4+000${i}_R.grb ICMGGfpp4+000${i} cdo -f nc copy ICMGGfpp4+000${i}01.grb ICMGGfpp4+000${i}_PLGG.nc cdo -f nc copy ICMGGfpp4+000${i}02.grb ICMGGfpp4+000${i}_MLGG.nc cdo splitzaxis ICMSHfpp4+000${i}.grb ICMSHfpp4+000${i} cdo sp2gpl ICMSHfpp4+000${i}01.grb ICMSHfpp4+000${i}01_GP.grb cdo sp2gpl ICMSHfpp4+000${i}03.grb ICMSHfpp4+000${i}03_GP.grb cdo sp2gpl ICMSHfpp4+000${i}04.grb ICMSHfpp4+000${i}04_GP.grb cdo -f nc copy ICMSHfpp4+000${i}03_GP.grb ICMSHfpp4+000${i}_ML.nc cdo -f nc copy ICMSHfpp4+000${i}01_GP.grb ICMSHfpp4+000${i}_PL.nc cdo -f nc copy ICMSHfpp4+000${i}04_GP.grb ICMSHfpp4+000${i}_surf.nc done
Possible problems and solutions
Workaround if CDO does not understand GRIB2 messages
If you are using an old version of CDO, or one that does not use the grib-api or ecCodes libraries, Use the grib_set command (from the grib_api installation) to change the GRIB edition number for all messages.
grib_set -s editionNumber=1 <input grib2 file> <output grib1 file>
Note this will only work if you are using less than 128 model levels as it does not actually convert the message to GRIB1 if there are any GRIB2 specific elements in the messages.
Parameter names are lost
The -t option tells CDO to use the predefined ECMWF parameter tables (see CDO documentation for more details). Without this option parameter names are not preserved. If the - t option doesn't work this is usually because CDO is using an old version of the ECMWF GRIB_API software.