The Observation Filter icon filters observation data from a BUFR file and produces output either as a BUFR file or as a geographical points (geopoints) file. You can use an Observation Filter as input to another Observation Filter as long as the first produces BUFR output.
From Macro/Python the icon can be called as obsfilter()
BUFR
BUFR stands for Binary Universal Form for Data Representation, the format in which observation data (in the broadest sense including ground observations, radiosonde soundings, radar and satellite images and soundings) are stored. Full details can be found in the WMO referencesand the ECMWF Meteorological Bulletin M1.4/4 "BUFR User Guide and Reference Manual" (1994).
Very (very) briefly, a BUFR file is composed of BUFR messages. Each message has header sections containing information (e.g. type, subtype, date and time) and a section containing one or more observations in a coded format. Each observation contains several values of different observed physical quantities (e.g. SYNOP, TEMP, PILOT, METAR,...). Observations are classified by Type (defined by WMO) and Subtype (which may be particular to each institution).
Geopoints
Geopoints files are ASCII files containing single or paired parameter (observed physical quantity) values together with space and time coordinates.
If you need observations of a given parameter for combination with other types of data, e.g. to derive differences between forecast fields and observations, they are required to be in geopoints format. You must use this icon to filter out the single parameter values out of the BUFR file or archived observation data and return/save them as a geopoints variable/file.
Filtering Efficiency
You may filter observations according to a wide variety of parameters or combinations thereof: you may filter on date and time, on location (meteorological station, WMO block, user defined area, proximity to user defined line) and range of values.
Regarding the structure of the input BUFR file, note that some of the filtering parameters such as observation type, subtype, date and time are located in the header part of the BUFR message, whilst others are located in the data part of the BUFR message itself.
This implies that the filtering of BUFR data according to parameters located in the header does not require decoding of the remaining information and thus is considerably (about 10 times) faster. Internally, filtering is always done first on the header parameters (if specified).
The Observation Filter Editor
Data
Drop any icon containing or returning BUFR data. This may be a MARS Retrieval (of BUFR) icon, a BUFR file icon or an Observation Filter icon (provided it outputs BUFR, not geopoints). The default icon is a MARS Retrieval for 4 day old 12 UTC synoptic observations for the whole globe.
Output
Specifies the output format you want. Specify one of the geopoints formats ( Geographical Points, Geographical Polar Vectors or Geographical X Y Vectors ) if you want to plot just one or two parameters or if you want to do calculations (including calculations with GRIB fields - combining geopoints with GRIB fields outputs geopoints). The parameters that follow - Parameter , Level , First Level , Second Level and Ocurrence Index - are not available if you specify BUFR output, as BUFR output must be formed by whole messages (of a given type).
Parameter
Available only for Output set to one of the geopoints formats (see above). To specify a parameter enter its unique descriptor value (a numerical code). Geographical Polar Vectors and Geographical XY Vectors require two descriptors, separated by a slash ( / ). The descriptor value is of the form XXYYY , where XX defines the class (e.g. 12 = Temperature class) and YYY the parameter within that class (e.g. 12004 = Dry bulb Temperature at 2m). These descriptor values are different from the ones of the GRIB format. If you do not know the descriptor value, click-left on the assist button to obtain a check list of parameters and associated descriptors. This list only contains the most common parameters. If the one you need is not on the list you have to look up its descriptor value in the "BUFR User Guide and Reference Manual" (ECMWF Meteorological Bulletin M1.4/4) - see "BUFR Table B", pages 99-108. The significance of code and flag values for non-quantitative parameters are given in the same reference, in "BUFR code table", pages 111-154.
Missing Data
Available only for Output set to one of the geopoints formats (see above). If set to Ignore , missing data is not included in the output file; this is the default behaviour. If set to Include , missing data will be output to the geopoints file, its value being set to that specified by Missing Data Value . Note that when the output format is one of the two geopoints vector formats, the test for missing data is only performed on the first parameter.
Missing Data Value
Available only for Output set to one of the geopoints formats and Missing Data set to Include . Any missing observations will be output as this value (default 0). It is wise, therefore, to ensure that this value is outwith the range of possible values for the requested parameter(s). Note that when the out- put format is one of the two geopoints vector formats, the test for missing data is only performed on the first parameter.
Level
Available only for Output set to one of the geopoints formats (see above). Specify one of Surface , Single Level , Layer and Occurrence . What you specify here must be consistent with the parameter you specified for filtering.
First Level
Available only for Output set to one of the geopoints formats (see above) and Level set to Single Level or Layer. Specify the level of the observation in hPa. If Layer was chosen for Level , the value will specify the bottom level of the layer. The assist button gives you a check list of most common pressure levels in hPa.
Second Level
Available only for Output set to one of the geopoints formats (see above) and Level set to Layer . Specify the top level of the layer in hPa.
Occurrence Index
Only available if Level set to Occurrence. Specify the numerical index of a parameter that has several values within one observation (e.g. cloud amount on different levels or water temperature at
different depths).
Observation Types
Specifies the numerical code or text string for the desired observation type. The assist button provides a partial list of available text strings and associated types. Observation types are standardised by WMO and are fixed from place to place. See the "BUFR User Guide and Reference Manual" (ECMWF Meteorological Bulletin M1.4/4) - "BUFR Table A", page 97 - for a complete list of numerical codes.
Observation Subtypes
Specifies the numerical code or text string for the desired observation subtype. The assist button provides a list of available numerical codes and associated subtypes. Note that institutions are free to define their own subtypes hence these are not an international standard.
Date
Specifies observation(s) date. It is not possible to specify a range of dates. If you are filtering a new MARS Retrieval, remember that archived observations are always a couple of days old - trying to retrieve yesterday’s observations is likely to fail. Allowed formats for the date are as follows :
- Absolute as YYYYMMDD
- Relative as -n ; n is the number of days before today ( -1 = yesterday)
- To specify a number of different dates use : YYYYMMDD1/YYYYMMDD2/...
Specifying a value for Date requires setting a value for Time (if both are set to ANY , changing Date will change Time from ANY to 12).
Time
Specifies time of the observation(s). Required format is HHMM. It is not possible to specify a range of times.
Resolution in Mins
Specifies a time window in minutes around the value chosen for Time .
WMO Blocks
Specifies a WMO block number. These identify a geographical region, e.g. 02 for Sweden and Finland, 16 for Italy and Greece.
WMO Stations
Specifies a list of WMO stations, using the five digit station identifier (the first two of which are the WMO block number).
Location Filter
Specifies one of No Location Filter , Area , Cross-Section Line. This allows you to filter observations contained within a geographical area or within a given proximity to a geographical line between two points.
Area
Specifies the coordinates of the area of interest. Enter coordinates (lat/long) of an area separated by a "/" (top left lat and long, bottom right lat and long); alternatively, use the coordinate assist button.
Line
Specifies the coordinates of a transect line. Enter coordinates (lat/long) of a line separated by a "/" (easternmost lat and long, westernmost lat and long); alternatively, use the coordinate assist button. This will filter all stations close enough to the line - how close is defined by Delta in Km .
Delta in Km
Specifies the width of the cross section line defined in Line.
Custom Filter
This allows you to filter observations of a given parameter according to its value. You can select observations equal to a value (Filter by Value) or within/outside a given range of values (Filter by Range / Filter by Exclude Range). Note that naturally you must specify one observed parameter to be filtered in this way.
Custom Parameter
Specifies the descriptor value of the parameter you want to filter according to value.
Custom Values
Specifies the desired numerical values for filtering by value. You may specify more than one value, separated by a forward slash (e.g. n1/n2 ). If you Filter by Value, observations of the selected parameter with the value equal to n1 or n2 are selected (you may specify more than two values). If you Filter by Range, observations of the selected parameter with the value within the n1 to n2 interval are selected. If you Filter by Exclude Range, observations of the selected parameter with the value outside the n1 to n2 interval are selected.