Versions Compared

Old Version 43

changes.mady.by.user Sandor Kertesz

Saved on

compared with

New Version Current

changes.mady.by.user Shahram Najm

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Apply the rules defined in rules_file to each BUFR message in the BUFR files provided as arguments. If you specify '-' (a single dash) for the rules_file, the rules will be read from standard input.

USAGE

bufr_filter [options] rules_file bufr_file bufr_file ...

OPTIONS


-f

Force. Force the execution not to fail on error.


-o output_grib_file

Output grib is written to output_grib_file. If an output grib file is required and -o is not used, the output grib is written to filtered'filter.out'


-V

Version.


-g

Copy GTS header.

...

Verbose.

bufr_filter examples

  1. The bufr_filter sequentially processes

    2. sequentially

  2. all bufr messages contained in the input files and applies the rules to each

    3. one

  3. of them. Input messages can be written to the output by using the "write" statement. The write statement can be parameterised so that output is sent to multiple files depending on key values used in the output file name.

    4. If

  4. First we write a rules_file containing the

    5. only

  5. following statement:

    Code Block
    write "../data/split/[bufrHeaderCentre:
    6. l
  6. i]_[dataCategory].bufr[editionNumber]";

    Then we create the BUFR file multitype.bufr from three others:

    Code Block
    > mkdir ../data/split 
> cat ../data/bufr/syno_1.bufr ../data/bufr/goes_87.bufr ../data/bufr/gosat.bufr > ../data/split/multitype.bufr

    Applying this rules_file to the "../data/

    7. bufr

  7. split/

    8. mutitype

  8. multitype.bufr" bufr file we obtain several files in the ../data/split directory containing messages split according to their key values

    Code Block
    > bufr_filter rules_file ../data/
    9. bufr
  9. split/multitype.bufr
> ls ../data/split
98_0.bufr3
98_3.bufr4
98_5.bufr3
    10. ecmf_3
  10. multitype.
    11. bufr4
  11. bufr

...



  2. The bufr header information can be accessed without unpacking the data. This rules_file:
     Code Block
    print "[bufrHeaderCentre] [bufrHeaderSubCentre] [masterTablesVersionNumber] [localTablesVersionNumber] [numberOfSubsets]";

    will result in the following output:
     Code Block
    > bufr_filter rules_file ../data/bufr/syno_multi.bufr
98 0 13 1 1
98 0 13 1 1
98 0 13 1 1

...


  2. To print values from the data section the messages have to be unpacked. To do that we need to set key 
    3.  unapack 
  3.  unpack to 1. This rules_file:
     Code Block
    set unpack=1;
print "block=[blockNumber] station=[stationNumber] lat=[latitude] lon=[longitude] t2=[airTemperatureAt2M]";

    will print out some data values from the specified SYNOP bufr messages.
     Code Block
    > bufr_filter rules_file ../data/bufr/syno_multi.bufr
block=1 station=1 lat=70.93 lon=-8.67 t2=274.5
block=1 station=3 lat=77 lon=15.5 t2=268.4
block=1 station=7 lat=78.92 lon=11.93 t2=268.5

 


  2. bufr_filter allows defining new keys with the transient keyword. We will further develop the previous example by creating a new key to combine the block number and the station number into the full WMO station id:
     Code Block
    set unpack=1;
transient statid=1000*blockNumber+stationNumber;
print "statid=[statid] lat=[latitude] lon=[longitude] t2=[airTemperatureAt2M]";

    The result is:
     Code Block
    > bufr_filter rules_file ../data/bufr/syno_multi.bufr
statid=1001 lat=70.93 lon=-8.67 t2=274.5
statid=1003 lat=77 lon=15.5 t2=268.4
statid=1007 lat=78.92 lon=11.93 t2=268.5

 


  2. We can use conditional statements in bufr_filter. The syntax is:
     Code Block
    if ( condition ) { block of rules } else { block of rules }

    The condition can be made using ==,!= and joining single block conditions with || and &&. The statement can be any valid statement also another nested condition The rules_file below shows 
    3.  us 
  3.  how to filter only SYNOP messages with a specific station id:
     Code Block
    set unpack=1;
transient statid=1000*blockNumber+stationNumber;

if (dataCategory ==0 && statid == 1003) {

    4. 	
  4.   write "out.bufr";
}

...


  2. The switch statement is an enhanced version of the if statement. Its syntax is the following:
     Code Block
    switch (key1
    3. ,key2,...,keyn
  3. ) {
    case 
    4. val11,val12,...,val1n
  4. val1:
        # 
    5. block of rules;
  5. statements
    case 
    6. val21,val22,...,val2n
  6. val2:
        # 
    7. block of rules;
  7. statements
    default:
        # 
    8. [ block of rules ]
  8. statements
}

    9. Each 
  9. The value of 
    10.  each 
  10. the key given as argument to the switch statement is matched against the values specified in the case statements. If there is a match, then the 
    11.  block or rules 
  11. statements corresponding to the matching case 
    12.  statement is 
  12. are executed. Otherwise, the default case is executed. The default case is mandatory 
    13. , even if empty
  13. if the case statements do not cover all the possibilities. The "~" operator can be used to match "anything".
 

  2. To access the keys' attributes use the -> operator. The example below prints the attributes of key 
    3.  pressure 
  3.  nonCoordinatePressure from a SYNOP bufr message.
     Code Block
    print "
    4. pressure
  4. nonCoordinatePressure=[
    5. pressure
  5. nonCoordinatePressure] [
    6. pressure
  6. nonCoordinatePressure->units]";
print "
    7. pressure
  7. nonCoordinatePressure->code=[
    8. pressure
  8. nonCoordinatePressure->code!06d]";
print "
    9. pressure
  9. nonCoordinatePressure->scale=[
    10. pressure
  10. nonCoordinatePressure->scale]";
print "
    11. pressure
  11. nonCoordinatePressure->reference=[
    12. pressure
  12. nonCoordinatePressure->reference]";
print "
    13. pressure
  13. nonCoordinatePressure->width=[
    14. pressure
  14. nonCoordinatePressure->width]";
print "
    15. pressure
  15. nonCoordinatePressure->percentConfidence=[
    16. pressure
  16. nonCoordinatePressure->percentConfidence] [
    17. pressure
  17. nonCoordinatePressure->percentConfidence->units]";
print "
    18. pressure
  18. nonCoordinatePressure->percentConfidence->code=[
    19. pressure
  19. nonCoordinatePressure->percentConfidence->code!06d]";
print "
    20. pressure
  20. nonCoordinatePressure->percentConfidence->scale=[
    21. pressure
  21. nonCoordinatePressure->percentConfidence->scale]";
print "
    22. pressure
  22. nonCoordinatePressure->percentConfidence->reference=[
    23. pressure
  23. nonCoordinatePressure->percentConfidence->reference]";
print "
    24. pressure
  24. nonCoordinatePressure->percentConfidence->width=[
    25. pressure
  25. nonCoordinatePressure->percentConfidence->width]";

    The result is:
     Code Block
    > bufr_filter rules_file ../data/bufr/syno_1.bufr

    26. pressure
  26. nonCoordinatePressure=100910 Pa

    27. pressure
  27. nonCoordinatePressure->code=010004

    28. pressure
  28. nonCoordinatePressure->scale=-1

    29. pressure
  29. nonCoordinatePressure->reference=0

    30. pressure
  30. nonCoordinatePressure->width=14

    31. pressure
  31. nonCoordinatePressure->percentConfidence=74 %

    32. pressure
  32. nonCoordinatePressure->percentConfidence->code=033007

    33. pressure
  33. nonCoordinatePressure->percentConfidence->scale=0

    34. pressure
  34. nonCoordinatePressure->percentConfidence->reference=0

    35. pressure
  35. nonCoordinatePressure->percentConfidence->width=7

 


  2. To access keys by rank  (i.e. by their occurrence in the message) use the # operator. The example below prints the value from the 4th occurrence of key pressure from a TEMP bufr message. As a reference, we also print all the pressure values found in the message.
     Code Block
    set unpack=1;
print "pressure=[
    3. pressure#4
  3. #4#pressure] [
    4. pressure#4
  4. #4#pressure->units]";
print "pressure=[pressure]";

    The result is:
     Code Block
    > bufr_filter rules_file ../data/bufr/temp_101.bufr
pressure=98500 Pa
pressure=102000 101800 100000 98500 96400 92500 92100 89700 
88100 86100 85000 84400 79400 79000 78300 77300 
71900 70000 69400 65100 61200 53400 50000 43900 
40000 39900 37800 31600 30000 27500 25000 21200 
21000 20600 20400 20000 19300 18400 17000 16600 
15100 15000 14600 14000 13400 13200 12900 11100 
10800 10000 8960 7630 7000 6420 6190 5770 
5320 5000 3970 3570 3190 3090 3000 2820 
2630 2400 2340 2050 2000 1680 1530 1500 
1380 1300 1210 31600

 


  2. It is possible to access elements by conditions  imposed on coordinate descriptors. The example below prints the temperature values on temperature significant levels from a TEMP bufr message. For temperature significant levels the key verticalSoundingSignificance=4 and this is what we use in the condition:
     Code Block
    set unpack=1;
print "[/verticalSoundingSignificance=4/airTemperature]";

    The result is:
     Code Block
    > bufr_filter rules_file ../data/bufr/temp_101.bufr
272.1 269.5 268.1 267.9 266.7 266.1 264.9 264.9 
260.5 260.9 263.5 263.7 261.7 261.9 259.1 258.9 
251.5 243.9 238.3 236.7 221.7 212.7 215.5 215.9 
214.1 217.3 218.3 217.3 219.3 218.9 219.5 217.9 
218.3 217.5 220.3 219.1 220.1 217.3 216.5 217.7 
215.9 217.1 213.5 216.1 214.7 216.1 215.3 216.5 
213.9 215.3 215.7 212.7 214.1 216.1 213.7 215.3 
214.9



  3. Another example for accessing keys by condition is to read scatterometer data. File asca_139.bufr contains a single message with 2016 subsets in a compressed form. In this case each subset has exactly the same structure: they store one location with several beams and one backscatter value in each beam. To print the backScatter values for beamIdentifier=2 from all the subsets we can simply define the condition like this:
     Code Block
    set unpack=1;
print "/beamIdentifier=2/backscatter=[/beamIdentifier=2/backscatter]";

    The result is:
     Code Block
    > bufr_filter rules_file ../data/bufr/asca_139.bufr
/beamIdentifier=2/backscatter=-24.6 -24.78 -24.92 -25.05 -25.04 -24.72 -23.83 -22.57 
-21.71 -21.76 -21.81 -20.97 -19.97 -19.01 -17.8 -16.22 
-14.67 -13.26 -12.02 -11.01 -9.84 -7.31 -8.76 -10.13 
-11.36 -12.58 -13.49 -13.87 -13.77 -13.44 -13.42 -13.58 
-13.92 -14.6 -15.36 -16.22 -17.11 -17.98 -18.56 -18.58 
-18.49 -18.45 -22.66 -22.99 -23.37 -23.85 -24.27 -24.57 
-24.54 -24.17 -23.96 -24.47 -24.53 -23.11 -21.62 -20.27 
-18.93 -17.42 -15.78 -14.13 -12.6 -11.35 -10.06 -7.38 
-8.57 -9.82 -11.43 -12.88 -13.83 -14.25 -14.21 -14.16 
-14.32 -14.44 -14.73 -15.21 -15.94 -17 -17.87 -18.64 

...
  1.  and many more values ......



  2. Accessing keys by condition provides the means to read individual subsets from uncompressed data. File synop_multi_subset.bufr contains a single message with several subsets in an uncompressed form. To access a given subset we can simply use key subsetNumber in the filter condition. The example below shows how to read the blockNumber, stationNumber and airTemperaturefor the first 3 subsets in the message:
     Code Block
    set unpack=1;
print "subsetNumber=1 blockNumber=[/subsetNumber=1/blockNumber] stationNumber=[/subsetNumber=1/stationNumber] airTempearture=[/subsetNumber=1/airTemperature]";
print "subsetNumber=2 blockNumber=[/subsetNumber=2/blockNumber] stationNumber=[/subsetNumber=2/stationNumber] airTempearture=[/subsetNumber=2/airTemperature]";
print "subsetNumber=3 blockNumber=[/subsetNumber=3/blockNumber] stationNumber=[/subsetNumber=3/stationNumber] airTempearture=[/subsetNumber=3/airTemperature]";

    The result is:
     Code Block
    > bufr_filter rules_file ../data/bufr/synop_multi_subset.bufr
subsetNumber=1 blockNumber=1 stationNumber=27 airTempearture=276.45
subsetNumber=2 blockNumber=1 stationNumber=84 airTempearture=266.55
subsetNumber=3 blockNumber=1 stationNumber=270 airTempearture=275.25