Page History

...

The tigge_check can do also some basic quality control by checking the allowed value ranges for each parameter (with -v option) if they were defined. There is another newer better maintainable tool doing similar basic quality check called grib_check.py. Read more information about both tools in Data quality checking tools (python source code is available there).

The gribThere is a further tool grib_enc_check.py can be also  which is used for data encoding check of another more recent project LC-WFV  (Lead Centre for Wave Forecast Verification). The encoding checking is not as comprehensive as in tigge_check e.g. geometry checks are missing completely.

Examples of tigge_check usage

...

Examples of grib_enc_check.py usage

grib_enc_check.py options

# BIN=/home/ma/emos/def/lcwfv/bin
python $BIN/grib_enc_check.py         
usage: grib_enc_check.py [-h] [-v VERBOSITY] [-wd DEFS]
  [-his] [-l [LISTING]]
                     [inp_file [inp_file ...]]

positional arguments:
  inp_file              enter input file name(s)

optional arguments:
  -h, --help            show this help message and exit
  -v VERBOSITY, --verbosity VERBOSITY
                        increase output verbosity [0-2]
  -w, --warning         warnings are treated as errors..
  -hisd DEFS, --historydefs DEFS      add value ranges history, if exists, to listing..
  -l [LISTING], --listing [LISTING]
                        list defined parameters (search if argument provided!)path to definition files

Checking LC-WVF data

$BIN/grib_enc_check.py lw.grib2
 field 223(Mean wave direction) key: dataRepresentationTemplateNumber expected: <0..2> encoded: 40
 field 224(10 metre U wind component) key: dataRepresentationTemplateNumber expected: <0..2> encoded: 40
Number of error(s) found: 2

List  defined limits for a parameter

python /home/ma/emos/def/lcwfv/bin/grib_check.py -l 131
131   [U component of wind]   [min <-250, 5> max <1, 250>] (default) 
                              [min <-150, -10> max <10, 150>] (levtype:pv) 
                              [min <-250, 5> max <-250, 250>] (class:s2, origin:rksl)

Performance tip to speed up checking big files 

There is a new tool  (ecCodes v>=2.6.0) called codes_split_file which is useful for parallellising decoding/checking tasks like tigge_check.

NAME    codes_split_file
DESCRIPTION
        Split an input file (GRIB, BUFR etc) into chunks of roughly the same size.

        The output files are named input_1, input_2 etc. This is much faster than grib_copy/bufr_copy.
USAGE
        codes_split_file [-v] nchunks input
OPTIONS
        -v  Print the count of messages and files created

If one has a very large input file with 1000s of messages, instead of running one process which sequentially checks each message in the file, one can split the file into 8 chunks and run the checking code in parallel on the 8 output files.

...