Format overview

Geopoints is the ASCII format used by Metview to handle spatially irregular data (e.g. observations). There are a number of variations on the format, but the default one is a 6-column layout. The columns do not have to be aligned, but there must be at least one whitespace character between each entry.

This example shows a geopoints file containing dry bulb temperature at 2m (PARAMETER = 12004).

Variants of the format allow 2-dimensional variables to be stored (e.g. U/V or speed/direction wind components), and another variant stores only lat, lon and value for a more compact file.

Examining geopoints

Examine the supplied geopoints.gpt icon to confirm the contents of the file. The columns are sortable. You may wish to open the file in an external text editor to see exactly what it looks like.

Visualising geopoints

Visualise the icon. The visdef used for geopoints is Symbol Plotting, and its default behaviour is to plot the actual numbers on the map. This can become cluttered, and text rendering can be slow. Create a new Symbol Plotting icon and set the following parameters:

Rename the icon to symb_auto and drop it into the Display Window to see the points coloured according to their value.

Computing some statistics in Macro

First, we will print some information about our geopoints data. Create a new Macro icon, type this code and run it:

Perform a simple data manipulation and return the result to Metview's user interface:

Save the macro and see its result by right-clicking on its icon and choosing examine or visualise. We could also have put a write() command into the macro to write the result to a geopoints file.

Finding geopoints points within 100km of a given location

As a more complex example, we will combine two functions in order to find the locations of the points within a certain distance of a given location. We will use the same geopoints file as before.

The distance() function returns a new geopoints variable based on its input geopoints, where each point's value has been replaced by the distance of that point from the given location. The description of this function follows:

Choose a location and use this function to compute the distances of the data points from it. Assign the result to a variable called distances and return it to the user interface to examine the numbers. The distances are in metres.

Now we will see a boolean operator in action. The expression distances < 100000 (one hundred thousand) will return a new geopoints variable where, for each point, if the input value was less than 100000, the resulting value will be 1; otherwise the resulting value will be zero. So the resulting geopoints will have a collection of ones and zeros. Confirm that this is the case.

The filter() function, from the documentation:

Use this in combination with what you have already done to produce a geopoints variable consisting only of the points within 100km of your chosen location. Plot the result to confirm it.

Saving geopoints data

Geopoints variables can be saved to disk using the write() command:

It is also possible to convert between geopoints and GRIB

This format - this will be covered in Processing Data.

Observation Data in BUFR files

Much observation data is received in BUFR format. BUFR is a complex format, capable of storing almost anything; BUFR files can vary widely, but there are some conventions which can help software to interpret them. We will have a brief overview of Metview's BUFR-handling capabilities here; for more information, see the dedicated tutorial on the Tutorials page.

Examining BUFR Meta-data

Right-click on the supplied synop.bufr BUFR icon and select examine from the icon menu. This will start the BUFR examiner application. The right-hand panel displays data for the message selected in the left-hand panel. This can be an easy way to find the correct descriptor for a given parameter such as Relative Humidity.

Plotting BUFR Data

Metview is able to plot certain BUFR data directly, mainly some WMO conventional observation types including SYNOP and TEMP.

Right-click and visualise the synop.bufr  BUFR icon. This will bring up the Display Window using the default visualisation assigned to observation plotting. What we see here is a spatially thinned set of SYNOP observations plotted on the map by using the official WMO-style. If you zoom into a smaller area you will see more observations but the thinning is still kept so that the plot should not seem cluttered.

Filtering Observation Data

BUFR files can contain a lot of information, but we often want to extract just one or two parameters.

The Observation Filter icon extracts a single scalar or vector value from each message in a BUFR file. It is able to perform filtering according to message type, date, time, level, area, location and custom descriptors. Examine the BUFR file and find the descriptor for wind speed at 10m (look in the blue right-hand panel) - make a note of it.

Create a new Observation Filter icon, rename it to wind_speed and edit it. Drop the BUFR icon into the Data field and set the following to extract the wind speed values in geopoints format:

Visualise the icon - the filtering will take place, then the result is plotted using the default Symbol Plotting definition, which is to plot the data as numbers. Drop your symb_auto icon into the Display Window for a nicer plot.

Notice that there is a point which claims a wind speed of 80m/s! Reliability can be a big issue with observational data, and this point claims winds of 288km/h! We can filter out data that we consider unrealistic - add the following parameters to your wind_speed icon:

This ensures that we only extract points whose wind speed is between 0 and 50 (m/s). Having a smaller range of values also allows the automatic colour range to spread more evenly through the data. There is still a point with a large value, which you can also filter out if desired.

Notice that the values in the colour scale change as you zoom in and out of different areas - this is computed according to the data currently visible. Try the supplied icon symb_wind_speed_fixed, which has a fixed value/colour mapping.

Extracting vector values from BUFR

We can extract the wind direction too, and plot the wind as arrows (or flags).

Make a copy of your wind_speed filter icon and call it wind_speed_and_direction. Find out which descriptor provides wind direction, then change the following parameters in your new filter icon:

where ????? is replaced by the wind direction descriptor number.

When you visualise the icon, you will see numbers as before, but if you drop a newly created Wind Plotting icon into the Display Window you will see wind arrows. Try the supplied coloured_wind_arrows icon too. Try changing it to plot wind flags instead of arrows.

You may wish to customise a Coastlines icon to provide a darker background for the plot.

Extra Tasks

Write a macro to plot the wind arrows from BUFR

Use the icons you created to filter and plot the wind arrows from BUFR data to write a macro which produces the same plot. Extract the 'magic numbers' such as the filtering threshold and the wind parameter descriptors into variables at the top of the macro, and use these variables in the macro rather than the raw numbers.

Investigate different grids

GRIB fields are often not as simple as regular lat/lon! ECMWF also produces data in "reduced Gaussian grids", two of which are included in your folder. Visualise them with your grid_1x1 icon to see how the points are spaced around the globe. Use both a cylindrical and a polar stereographic projection to look at them (Geographic View icon).

Try the search facilities in the data examiners

Examine the GRIB file and the BUFR file; press CTRL-F to initiate the search. Look carefully at the options!

Create your own GRIB Examiner key profile

When you examine a GRIB file, a list of 'keys' is used to display the GRIB messages - one key per column. These columns are configurable - a 'key profile' is a set of keys, and you can create as many of them as you want. It can be very useful to have different key profiles for different tasks. From the user interface in the GRIB Examiner, create a new key profile; starting either from scratch, or else from a duplicate of the default profile. Note that the Display Window also operates on the same principles, and you can share key profiles between the two.

Observation filtering

Extract 2m temperature values below the freezing point from synop.bufr.

Hints:

• use geopoints output
• use custom filter
• temperature values are given in K