You are viewing an old version of this page. View the current version.
Compare with Current
View Page History
« Previous
Version 8
Next »
For the projects TIGGE, S2S and UERRA the exact data format, WMO compliant GRIB2, is required to allow easy data processing and intercomparison. Read in Data encoding checking tools more information about tigge_check and grib_check.py tools. Apart from encoding verification, theses tools can do also some basic quality control by checking the allowed value ranges, if they were defined, for each parameter.
Please be aware that some previously allowed valid ranges can become obsolete at some point e.g. due to model upgrade to higher resolution meaning in general different values of some parameters. Because of that a continuous set up updates are needed so it is important to use the latest versions of the tools. Also for some parameters the allowed ranges must not be defined due to various reasons. In some cases the allowed limits for different data sets or data origins might also exist because of known special features or deficiencies of some related model's outputs.
module load eccodes
load eccodes 2.9.2 Library (ECCODES_DIR, ECCODES_VERSION, ECCODES_INCLUDE, ECCODES_LIB, ECCODES_INCLUDE_DIR, ECCODES_LIB_DIR)
# example with one data value ranges warning
tigge_check -u -w -v
tigge_mogreps-g_2019021600_00.grib2
warning: tigge_mogreps-g_2019021600_00.grib2, field 513 (Potential vorticity, paramId=60), matched definition (class:ti, model:glob, origin:egrr)
=> maximum value 1296779.249971 is not in [0.000001,100.000000] |
export BIN=/home/ma/emos/def/s2s/s2s_prod/bin
python $BIN/grib_check.py lw.grib2
warning: lw.grib2, field 700 (Peak wave period, paramId=140231), matched definition (default) => minimum value -9.699028 is not in [0.000000,35.000000]
warning: lw.grib2, field 718 (Peak wave period, paramId=140231), matched definition (default) => maximum value 41.463626 is not in [0.000000,35.000000] |
List all defined limits
python $BIN/grib_check.py -l
165 [10 metre U wind component] [min <-150, -0.1> max <0.1, 150>] (default)
[min <-150, -0.1> max <0.1, 150>] (class:ti, model:lam, suiteName:mogreps-mo-eua)
166 [10 metre V wind component] [min <-100, -1> max <0.5, 100>] (default)
[min <-100, 10> max <-10, 100>] (class:ti, model:lam, suiteName:mogreps-mo-eua)
260260 [10 metre wind direction] [min <0, 1> max <359, 360.1>] (default)
...
|
See allowed limits for parameters with paramId containing 141
python $BIN/grib_check.py -l 141
228141 [Snow depth water equivalent] [min <-1e-10, 0> max <100, 15000>] (default)
[min <-1e-10, 0> max <100, 40000>] (class:s2, origin:cwao)
[min <-1, 0> max <100, 100000>] (class:ti) |
See allowed limits for parameters with paramId containing 141 including past set up history
python $BIN/grib_check.py -l -his 141
228141 [Snow depth water equivalent] [min <-1e-10, 0> max <100, 15000>] (default)
# History of changes:
#
# 1.version (valid from: 2017-01-01)
# min <-1e-10, 0>
# max <100, 15000>
# info:See long history of changes in tigge_check.h!
228141 [Snow depth water equivalent] [min <-1e-10, 0> max <100, 40000>] (class:s2, origin:cwao)
# History of changes:
#
# 1.version (valid from: 2013-02-11)
# min <-1e-10, 0>
# max <100, 40000>
# info:s2s.z_s2s_c_cwao_20130211000000_glob_prod_sl_000.grib2, field 23: snow_depth_water_equivalent_sfc.glob.s2 maximum value 30452.5 is not in [100,15000]
228141 [Snow depth water equivalent] [min <-1, 0> max <100, 100000>] (class:ti)
# History of changes:
#
# 3.version (valid from: 2017-08-08)
# min <-1, 0>
# max <100, 100000>
# info:z_tigge_c_RKSL_20170808000000_glob_prod_cf_0240_000.grib2, field 119 (Snow depth water equivalent, paramId=228141), matched definition (default) => maximum value
94537.8 is not in [100,15000]
# 2.version (valid from: 2017-08-08)
# min <-1, 0>
# max <100, 90000>
# info:warning: tigge_mogreps-g_2017080812_00.grib2, field 445 => maximum value 67640.3 is not in [100,15000]
# 1.version (valid from: 2017-10-17)
# min <-1, 0>
# max <100, 90000>
# info:warning: ncmrwf_tigge_20170807_000.grib2, field 1749 (Snow depth water equivalent, paramId=228141), matched definition (default) => maximum value 78601.367188 is
not in [100.000000,15000.000000] |
The prerequisite is to have eccodes python modules available in the system. The tool itself consist of two python scripts attached to this page:
The set up file contains mostly a python dictionary with the allowed ranges for parameters. These ranges can be easily tuned and can be specific for specific selection of metadata if needed (e.g. MARS class, level type etc).
Please be aware that this tool is not part of any package yet as is still under development. For example some code refactoring or performance optimisation might be needed. 