Documentation

The ecCodes GRIB tools are documented at:

There are examples of how to use them at:

Other useful resources:

Preliminaries

Login to ecs-login and work in your $SCRATCH:

$> cd $SCRATCH

Make a copy of the grib_decoding directory in your $SCRATCH to Atos:

$> tar -xvf /home/trx/grib_decoding.tar

This will create a directory in your $SCRATCH called grib_decoding containing the GRIB data files for this tutorial.

There are sub-directories for each practical:

$> ls $SCRATCH/grib_decoding

demo inspect  modify  python

For this tutorial cd to the inspect directory:

$> cd $SCRATCH/grib_decoding/inspect


Remember to load the ecmwf-toolbox !

$> module load ecmwf-toolbox

This tutorial covers the following topics:

Using grib_dump and grib_ls to inspect GRIB files

Use grib_ls to inspect the content of the files t2m.grib1 and t2m.grib2

What fields do the GRIB messages contain ?

To list the GRIB messages in t2m.grib1

$>  grib_ls t2m.grib1
t2m.grib1
edition      centre       typeOfLevel  level        dataDate     stepRange    dataType     shortName    packingType  gridType     
1            ecmf         surface      0            20251008     0            cf           2t           grid_simple  reduced_gg  
1            ecmf         surface      0            20251008     6            cf           2t           grid_simple  reduced_gg  
1            ecmf         surface      0            20251008     12           cf           2t           grid_simple  reduced_gg  
1            ecmf         surface      0            20251008     18           cf           2t           grid_simple  reduced_gg  
1            ecmf         surface      0            20251008     24           cf           2t           grid_simple  reduced_gg  
5 of 5 messages in t2m.grib1

5 of 5 total messages in 1 files

The file contains 2 metre temperature (shortName=2tl) on a reduced Gaussian grid (gridType=reduced_gg) from the ECMWF Ensemble control forecast (dataType=cf) for 6-hourly steps (stepRange) from 0 to 24 hours for dataDate=20251008.

Similarly, for t2m.grib2:

$>  grib_ls t2m.grib2
t2m.grib2
edition      centre       date         dataType     gridType     stepRange    typeOfLevel  level        shortName    packingType  
2            ecmf         20251008     cf           reduced_gg   0            heightAboveGround  2            2t           grid_simple 
2            ecmf         20251008     cf           reduced_gg   6            heightAboveGround  2            2t           grid_simple 
2            ecmf         20251008     cf           reduced_gg   12           heightAboveGround  2            2t           grid_simple 
2            ecmf         20251008     cf           reduced_gg   18           heightAboveGround  2            2t           grid_simple 
2            ecmf         20251008     cf           reduced_gg   24           heightAboveGround  2            2t           grid_simple 
5 of 5 messages in t2m.grib2

5 of 5 total messages in 1 files

Again, the file contains 2 metre temperature (shortName=2t) on a reduced Gaussian grid (gridType=reduced_gg) from the ECMWF Ensemble control forecast (dataType=cf) for 6-hourly steps (stepRange) from 0 to 24 hours for dataDate=20170217.

In fact, they are the same fields as in t2m.grib1 but encoded in GRIB edition 2 and obtained from the TIGGE archive.

Do the default keys printed by grib_ls depend on the GRIB edition ?  Do you see any other differences ?

No, grib_ls prints the same set of default keys for the both GRIB edition 1 and GRIB edition2. However, the order of the columns output is different.

The only differences seen with grib_ls is for the value of the typeOfLevel and level keys which are

  • typeOfLevel=surface and level=0 inthe GRIB1 encoding; and
  • typeOfLevel=heightAboveGround and level=2 for the GRIB2 encoding.

How do you print the MARS keys for the two files ?

To print the mars keys, use either grib_ls -m or grib_ls -n mars.  This is the output for the file t2m.grib1:

$> grib_ls -m t2m.grib1
t2m.grib1
domain       levtype      date         time         step         param        class        type         stream       expver       _leg_number  
g            sfc          20251008     0000         0            167.128      od           cf           enfo         0001         1           
g            sfc          20251008     0000         6            167.128      od           cf           enfo         0001         1           
g            sfc          20251008     0000         12           167.128      od           cf           enfo         0001         1           
g            sfc          20251008     0000         18           167.128      od           cf           enfo         0001         1           
g            sfc          20251008     0000         24           167.128      od           cf           enfo         0001         1           
5 of 5 messages in t2m.grib1

5 of 5 total messages in 1 files

For t2m.grib2, the same command gives:

$> grib_ls -m t2m.grib2
t2m.grib2
origin      date        time        step        levtype     number      param       expver      class       model       type        stream      
ecmf        20251008    0000        0           sfc         0           167         prod        ti          glob        cf          enfo       
ecmf        20251008    0000        6           sfc         0           167         prod        ti          glob        cf          enfo       
ecmf        20251008    0000        12          sfc         0           167         prod        ti          glob        cf          enfo       
ecmf        20251008    0000        18          sfc         0           167         prod        ti          glob        cf          enfo       
ecmf        20251008    0000        24          sfc         0           167         prod        ti          glob        cf          enfo       
5 of 5 messages in t2m.grib2

5 of 5 total messages in 1 files

How do you add the shortName to the output of the previous command ?

To print the shortName along with the list of mars keys use the –P option e.g.:

$> grib_ls -m -P shortName t2m.grib2
t2m.grib2
shortName   origin      date        time        step        levtype     number      param       expver      class       model       type        stream      
2t          ecmf        20251008    0000        0           sfc         0           167         prod        ti          glob        cf          enfo       
2t          ecmf        20251008    0000        6           sfc         0           167         prod        ti          glob        cf          enfo       
2t          ecmf        20251008    0000        12          sfc         0           167         prod        ti          glob        cf          enfo       
2t          ecmf        20251008    0000        18          sfc         0           167         prod        ti          glob        cf          enfo       
2t          ecmf        20251008    0000        24          sfc         0           167         prod        ti          glob        cf          enfo       
5 of 5 messages in t2m.grib2

5 of 5 total messages in 1 files

How do you change the previous command so that the output is in descending step order ?

Hint:  think about strings and integers

To order the output in descending step order use the –B option:

$> grib_ls -B "step:i desc" t2m.grib2
edition      centre       date         dataType     gridType     stepRange    typeOfLevel  level        shortName    packingType  
2            ecmf         20251008     cf           reduced_gg   24           heightAboveGround  2            2t           grid_simple 
2            ecmf         20251008     cf           reduced_gg   18           heightAboveGround  2            2t           grid_simple 
2            ecmf         20251008     cf           reduced_gg   12           heightAboveGround  2            2t           grid_simple 
2            ecmf         20251008     cf           reduced_gg   6            heightAboveGround  2            2t           grid_simple 
2            ecmf         20251008     cf           reduced_gg   0            heightAboveGround  2            2t           grid_simple

Note that you need to specify “step:i” so that step is treated as an integer rather than a character string.

How do you use grib_ls to print only the centre, dataDate, stepRange, typeOfLevel and shortName ?

To specify a set of keys to print, use the -p option:

$> grib_ls -p centre,dataDate,stepRange,typeOfLevel,shortName t2m.grib2
t2m.grib2
centre       dataDate     stepRange    typeOfLevel  shortName    
ecmf         20251008     0            heightAboveGround  2t          
ecmf         20251008     6            heightAboveGround  2t          
ecmf         20251008     12           heightAboveGround  2t          
ecmf         20251008     18           heightAboveGround  2t          
ecmf         20251008     24           heightAboveGround  2t          
5 of 5 messages in t2m.grib2

5 of 5 total messages in 1 files

How do you change the previous command so that only the field for forecast step=6 is printed ?

To print the keys for forecast step 6 only, use the -p option with the -w option:

$> grib_ls -w stepRange=6 -p centre,dataDate,stepRange,typeOfLevel,shortName t2m.grib2
t2m.grib2
centre       dataDate     stepRange    typeOfLevel  shortName    
ecmf         20251008     6            heightAboveGround  2t          
1 of 5 messages in t2m.grib2

1 of 5 total messages in 1 files

Using the same commands with the file t2m.grib1 gives similar output.

How do you change the previous command to print the centre as both a string and an integer ?

Specifying the key as "centre:i" and “centre:s” prints the centre both as an integer and a string:

$> grib_ls -w stepRange=6 -p centre:i,centre:s,dataDate,stepRange,typeOfLevel,shortName t2m.grib2
t2m.grib2
centre       centre       dataDate     stepRange    typeOfLevel  shortName    
98           ecmf         20251008     6            heightAboveGround  2t          
1 of 5 messages in t2m.grib2

1 of 5 total messages in 1 files

As before, using the same commands with the file t2m.grib1 gives similar output.

How do you use grib_dump to inspect the fourth (count=4) GRIB message in both files in WMO Octet mode ?

To use grib_dump to inspect the contents of the fourth message in t2m.grib1 and t2m.grib2 use the -w option with "count=4"  (the count key counts the number of messages starting form count=1 for the first message.  To print the output in WMO Octet mode use the -O option:

$> grib_dump –w count=4 -O t2m.grib1
...
$> grib_dump –w count=4 -O t2m.grib2
...

The command may produce many lines of output.  You may want to redirect output to a file using, e.g.:

$> grib_dump .... t2m.grib1 > t2m.grib1.output

To use grib_dump to all of the keys for the fourth message in JSON mode, use the -j option:

$> grib_dump -j -w count=4 t2m.grib2 > t2m.grib2.output

$> head -31 t2m.grib1.output
head -31 t2m.grib2.output
{ "messages" : [
  [

    {
      "key" : "discipline",
      "value" : 0
    },
    {
      "key" : "editionNumber",
      "value" : 2
    },
    {
      "key" : "centre",
      "value" : 98
    },
    {
      "key" : "subCentre",
      "value" : 0
    },
    {
      "key" : "significanceOfReferenceTime",
      "value" : 1
    },
    {
      "key" : "dataDate",
      "value" : 20251008
    },
    {
      "key" : "dataTime",
      "value" : 0
    },

Similarly, for t2m.grib1:

$> grib_dump -j -w count=4 t2m.grib1 > t2m.grib1.output

$> head -31 t2m.grib1.output
{ "messages" : [
  [

    {
      "key" : "editionNumber",
      "value" : 1
    },
    {
      "key" : "table2Version",
      "value" : 128
    },
    {
      "key" : "centre",
      "value" : 98
    },
    {
      "key" : "generatingProcessIdentifier",
      "value" : 158
    },
    {
      "key" : "indicatorOfParameter",
      "value" : 167
    },
    {
      "key" : "indicatorOfTypeOfLevel",
      "value" : 1
    },
    {
      "key" : "level",
      "value" : 0
    },


Using grib_ls to find values at the nearest grid point to a specified latitude-longitude point

The file t2m.grib1 contains the 2m temperature from the ENS control forecast at 6-hourly time steps for the first 24 hours on the O640 octahedral reduced Gaussian grid.  The file lsm.grib1 contains the ENS land-sea mask on the O640 grid.

How do you use grib_ls to find the value of the 2m temperature at the grid point nearest to ECMWF's headquarters in the UK (Lat 51.42°N, Lon 0.95°W) at each forecast step ?

Be careful to specify the longitude correctly !

To list the nearest points to Lat 51.42° N, Lon 0.95° W use:

$> grib_ls -l 51.42,-0.95 t2m.grib2
t2m.grib2
t2m.grib2
edition      centre       date         dataType     gridType     stepRange    typeOfLevel  level        shortName    packingType         value1  value2  value3  value4 
2            ecmf         20251008     cf           reduced_gg   0            heightAboveGround  2            2t           grid_simple  284.415 284.165 284.403 284.114 
2            ecmf         20251008     cf           reduced_gg   6            heightAboveGround  2            2t           grid_simple  285.711 285.787 285.647 285.621 
2            ecmf         20251008     cf           reduced_gg   12           heightAboveGround  2            2t           grid_simple  290.361 290.242 290.222 289.74 
2            ecmf         20251008     cf           reduced_gg   18           heightAboveGround  2            2t           grid_simple  286.573 286.94 286.475 286.282 
2            ecmf         20251008     cf           reduced_gg   24           heightAboveGround  2            2t           grid_simple  282.333 282.247 282.268 281.671 
5 of 5 messages in t2m.grib2

5 of 5 total messages in 1 files
Input Point: latitude=51.42  longitude=-0.95
Grid Point chosen #2 index=156197 latitude=51.38 longitude=359.03 distance=4.79 (Km)
Other grid Points
- 1 - index=156198 latitude=51.38 longitude=359.35 distance=21.64 (Km)
- 2 - index=156197 latitude=51.38 longitude=359.03 distance=4.79 (Km)
- 3 - index=155082 latitude=51.52 longitude=359.35 distance=23.65 (Km)
- 4 - index=155081 latitude=51.52 longitude=359.03 distance=11.09 (Km)

Note a longitude of 0.95° W must be specified as -0.95 !


Note that ecCodes converts all longitude values so that they lie in the interval 0° to 360°. In GRIB 2 all longitude values must be positive !

From the output of the previous command, what is the latitude and longitude of the grid point nearest to ECMWF's Headquarters in Reading, UK ?

The nearest grid point is at latitude=51.38° longitude=359.03° (0.77° W) and is 4.79 km from ECMWF.

How do you change the previous command to output only the forecast step and the 2m temperature value at the grid point nearest to ECMWF ?

To output only the forecast step and 2 metre temperature value at the nearest grid point set the MODE option to 1 and specify that the stepRange key only should be printed using the -p option:

$> grib_ls -l 51.42,-0.95,1 -p stepRange t2m.grib1
t2m.grib2
stepRange    value 
0           284.165     
6           285.787     
12          290.242     
18          286.94      
24          282.247     
5 of 5 messages in t2m.grib2

5 of 5 total messages in 1 files
Input Point: latitude=51.42  longitude=-0.95
Grid Point chosen #2 index=156197 latitude=51.38 longitude=359.03 distance=4.79 (Km)
Other grid Points
- 1 - index=156198 latitude=51.38 longitude=359.35 distance=21.64 (Km)
- 2 - index=156197 latitude=51.38 longitude=359.03 distance=4.79 (Km)
- 3 - index=155082 latitude=51.52 longitude=359.35 distance=23.65 (Km)
- 4 - index=155081 latitude=51.52 longitude=359.03 distance=11.09 (Km)

How would you change the previous command to specific that the land-sea mask in the lsm.grib1 file is used to identify the nearest land point and print the 2m temperature values at the nearest grid point in decimal format with 2 decimal places ?

The output format of the  values is specified with the -F option.  Here, -F"%.2f" is used to specify decimal format with 2 decimal places.

To specify the land-sea mask, pass the name of the file containing the land-sea mask as the FILE option:

$> grib_ls -F"%.2f" -l 51.42,-0.95,1,lsm.grib2 -p step  t2m.grib2 
t2m.grib2
step         value 
0           284.11      
6           285.62      
12          289.74      
18          286.28      
24          281.67      
5 of 5 messages in t2m.grib2

5 of 5 total messages in 1 files
Input Point: latitude=51.42  longitude=-0.95
Grid Point chosen #4 index=155081 latitude=51.52 longitude=359.03 distance=11.09 (Km)
Mask values:
- 1 - index=156198 latitude=51.38 longitude=359.35 distance=21.64 (Km) value=1.00
- 2 - index=156197 latitude=51.38 longitude=359.03 distance=4.79 (Km) value=1.00
- 3 - index=155082 latitude=51.52 longitude=359.35 distance=23.65 (Km) value=1.00
- 4 - index=155081 latitude=51.52 longitude=359.03 distance=11.09 (Km) value=0.98

Here the value=1.00 (or 0.98) at the end of the line showing the four nearest grid points indicates which are land points (mask=1.0). Land points have mask ≥ 0.5; sea points have mask<0.5.

Once the index of the nearest grid point is known, its value can be obtained directly with the -i option:

$> grib_ls -F"%.2f" -i 156197 -p step  t2m.grib2                 
t2m.grib2
step                value(156197) 
0           284.16      
6           285.79      
12          290.24      
18          286.94      
24          282.25      
5 of 5 messages in t2m.grib2

5 of 5 total messages in 1 files

This is faster than searching for the nearest grid point for each field.

Using grib_get_data to print data values

How do you print the latitude, longitude and values for the field in surface2.grib with the values output in decimal format with 5 decimal places  ?

To print the data values for surface.grib2 in decimal format with 6 decimal places use grib_get_data with the option -F"%.6f":

$> grib_get_data -F"%.6f" surface.grib2
Latitude Longitude Value
   60.000  -10.000 284.736931
   60.000   -7.500 284.633891
   60.000   -5.000 285.484602
   60.000   -2.500 285.914825
   60.000    0.000 285.845217
   60.000    2.500 286.129910
   60.000    5.000 287.123160
   ...

How do you change the previous command to also output the latitude and longitude with 6 decimal places ?

To output also the latitude and longitude values with 6 decimal places, we need to use the -L option:

$> grib_get_data -L"%.6f %.6f" -F"%.6f" surface.grib2
Latitude Longitude Value
60.000000 -10.000000 284.736931
60.000000 -7.500000 284.633891
60.000000 -5.000000 285.484602
60.000000 -2.500000 285.914825
60.000000 0.000000 285.845217
60.000000 2.500000 286.129910
60.000000 5.000000 287.123160
57.500000 -10.000000 285.959885
57.500000 -7.500000 286.499073
...

Note that we need to specify the format for both latitude and longitude separately:  -L"%.6f %.6f".

How can you find out whether there are any missing values in the field in surface1.grib ?

Missing values can be identified by using grib_get_data with the –m option, for example:

$> grib_get_data –m MISSING surface.grib2
Latitude, Longitude, Value
...
   60.000    5.000 2.8712316036e+02
   57.500  -10.000 2.8595988464e+02
   57.500   -7.500 2.8649907303e+02
   57.500   -5.000 MISSING
   57.500   -2.500 MISSING
   57.500    0.000 2.8624914455e+02
   57.500    2.500 2.8585043049e+02
   57.500    5.000 2.8566167641e+02
   55.000  -10.000 2.8659341431e+02
...

You can also print the key numberOfMissing with the grib_get or grib_ls tools:

$> grib_get -p numberOfMissing surface.grib2
26

So there are 26 missing values in total.

Other keys also provide the numberOfPoints and numberOfValues:

$> grib_get -p numberOfPoints,numberOfValues,numberOfMissing surface.grib2
63 37 26

So that:

numberOfPoints = numberOfValues + numberOfMissing