Metview's Python interface provides access to all of Metview's Macro language functions, with automatic translations between data types. Here's an example that shows how to retrieve model and observation data and compute their difference, and the weighted mean value of that difference, both in Macro and in Python.

# Metview Macro

t2m_fc48 = retrieve(
    type    : "fc",
    levtype : "sfc",
    param   : "2t",
    date    : -5,
    step    : 48,
    grid    : "o1280")

synop = retrieve(
    type   : "ob",
    repres : "bu",
    date   : -3)

synop_t2m = obsfilter(
    output    : "geopoints",
    parameter : 012004,
    data      : synop)

diff = t2m_fc48 - synop_t2m
print(integrate(diff))

import metview as mv

t2m_fc48 = mv.retrieve(
    type    = "fc",
    levtype = "sfc",
    param   = "2t",
    date    = -5,
    step    = 48,
    grid    = "o1280")

synop = mv.retrieve(
    type   = "ob",
    repres = "bu",
    date   = -3)

synop_t2m = mv.obsfilter(
    output    = "geopoints",
    parameter = "012004",
    data      = synop)

diff = t2m_fc48 - synop_t2m
print(mv.integrate(diff))

You can see that the Macro functions are all available through the namespace defined in the import command.

More examples are available in the Gallery.

Variable types

When calling Macro functions from Python, variables passed as input to or output from those functions undergo a conversion as described in this table:

Working with Fieldsets

Fieldsets work much the same as they do in the Macro language, but watch out for these things:

• length of a fieldset can be found with the len function: num_fields = len(my_fieldset)
• indexing starts at 0: first_field = my_fieldset[0]
• slicing works: my_fields = fs[0:6:2]
• you can pass a numpy array of indexes: my_fields = fs[np.array([1.0, 2.0, 0.0, 5.0])]
• comparison operators work the same as in Macro, i.e. they return a fieldset of 1s and 0s: smaller = fs1 < fs2
• equality and non-equality operators are == and !=
• Fieldsets can be directly constructed either as empty, or with a path to a GRIB file:
  • f = mv.Fieldset()
  • f = mv.Fieldset(path='test.grib')
    
• concatenation can be done like this:  my_fieldset.append(my_other_fieldset)
• iteration works: for f in my_fieldset:  #do something
  

Working with Geopoints

Geopoints also work much the same as they do in Macro, but be aware of these points:

• in Macro, we use geo_missing_value to denote missing data values; in Python, we use numpy.nan

Working with dates

There are several differences between the usage of the date object in Macro and the datetime object in Python. You will find a few examples below to compare the various date manipulation techniques used in Macro and Python, respectively.

# Metview Macro



# creating
d = 2000-01-04 09:50:24 
today = date(0)
yesterday = date(-1) 

# arithmetic
d = d + hour(9) + minute(50) + second(24) 


# using an increment of 2 days
for d = 2018-11-01 to 2018-11-10 by 2 do
  (...)
end for 
 



# using an increment of 6 hours
for d = 2018-11-01 to 2018-11-10 by hour(6) do
   x = retrieve(
      date : yymmdd(d),
      time : hhmm(d),
      ...)
   (...)
end for

import metview as mv
import numpy as np
from datetime import datetime, timedelta

# creating 
d = datetime(2000, 1, 4, 9, 50, 24)
today = datetime.today()
yesterday = datetime.today() - timedelta(1) 

# arithmetic
d = d + timedelta(hours=9, minutes=50, seconds=24) 


# using an increment of 2 days
d0 = datetime(2018,11,1)
d1 = datetime(2018,11,10)
dt = timedelta(days = 2)
for d in np.arange(d0, d1, dt):
    (...)


# using an increment of 6 hours
d0 = datetime(2018, 11, 1)
d1 = datetime(2018, 11, 10)
dt = timedelta(hours = 6)
for d = np.arange(d0, d1, dt):
	x = mv.retrieve(date =d, 
			time = mv.hour(d), ...)	
	(...)	

Additional data export features

NumPy arrays

Any Metview function that normally returns a vector will return a numPy array when called from Python. dtypes of float32 and float64 are supported. For example, the following fieldset functions return numPy arrays:

a = mv.read('my_data.grib') # returns a Fieldset
lats = mv.latitudes(a)      # returns a numPy array
lons = mv.longitudes(a)     # returns a numPy array
vals = mv.values(a)         # returns a numPy array

Pandas Dataframes

The Geopoints data type has an additional function, to_dataframe(), which can be used to produce a Pandas Dataframe object as shown:

import metview as mv
import pandas as pd

gpt = mv.read("gpts.gpt") # returns a Geopoints
df = gpt.to_dataframe()   # returns a Pandas Dataframe
print(df.head())

Output:

                 date  latitude   level  longitude    value
0 2018-01-14 12:00:00      30.0  1000.0      -24.0  288.736
1 2018-01-14 12:00:00      30.0  1000.0      -18.0  288.736
2 2018-01-14 12:00:00      30.0  1000.0      -12.0  286.736
3 2018-01-14 12:00:00      30.0  1000.0       -6.0      NaN
4 2018-01-14 12:00:00      30.0  1000.0        0.0      NaN

Xarray Datasets

The Fieldset object has an additional method, to_dataset(), which produces an xarray Dataset object from the given fieldset. This is an N-dimensional data array based on the Common Data Model used in netCDF. For example:

import metview as mv

t2m_fc = mv.retrieve(
	type    = 'fc',
	levtype = 'sfc',
	param   = ['2t', '2d'],
	date    = -5,
	step    = list(range(0, 48+1, 6)),
	grid    = [1,1]
)

xa = t2m_fc.to_dataset()
print(xa)

will produce the following output:

<xarray.Dataset>
Dimensions:    (latitude: 181, longitude: 360, step: 9, time: 1)
Coordinates:
  * time       (time) datetime64[ns] 2018-05-10T12:00:00
  * step       (step) timedelta64[ns] 0 days 00:00:00 0 days 06:00:00 ...
  * latitude   (latitude) float64 90.0 89.0 88.0 87.0 86.0 85.0 84.0 83.0 ...
  * longitude  (longitude) float64 0.0 1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0 9.0 ...
Data variables:
    2t         (time, step, latitude, longitude) float32 ...
    2d         (time, step, latitude, longitude) float32 ...
Attributes:
    Conventions:  CF-1.7
    comment:      GRIB to CF translation performed by xarray-grib

This is based on the Open Source cfgrib package which is planned to be ultimately integrated into xarray.

From version 0.8.6 of Metview's Python interface, it is also possible to convert an xarray dataset to a Metview Fieldset, and also to pass one directly to Metview functions. Note that this uses an experimental feature of cfgrib and will only work for a small subsset of xarray datasets. Examples:

grib = mv.read('temperature_on_pl.grib')
x = grib.to_dataset() # convert from Fieldset to dataset
f = mv.dataset_to_fieldset(x) # convert back to Fieldset

grib = mv.read('temperature_on_pl.grib')
x = grib.to_dataset() # x is now an xarray dataset
fs = mv.mean(x*2) # *2 is done by xarray, mean() is done by Metview

Icon functions

Macro functions which correspond to icons, such as retrieve(), which corresponds to the Mars Retrieval icon, can take their arguments in a number of ways:

Object-oriented calling

Metview's Python interface also provides a more object-oriented way of using the wrapper classes such as Fieldset. The following two snippets of code are equivalent:

Calling user-defined Macro functions from Python

Any user-defined Macro function can be called from Python as long as it satisfies both of these criteria:

1. the function must reside in a file of the same name as the function
2. the file must be in a directory listed in the METVIEW_MACRO_PATH environment variable; this variable can be modified by the user, but by default it will include $HOME/metview/System/Macros.

If these are satisfied, then the function may be invoked using the call() command as shown here:

import metview as mv
mv.call('my_func', 4, 1, 'my_string_arg'))

The first argument is the name of the function, the subsequent arguments will be passed to the function.

Things to watch out for

Although it's easy to convert an existing macro into a Python script, there are some things to be aware of.

• Macro indexing starts at 1, but Python indexing starts at 0. Aside from the standard Python structures such as lists, this is also true for Metview Python classes such as Fieldset. Given a fieldset fs, the first field is obtained in Macro by fs[1], but in Python it is obtained by fs[0].
• Some MARS requests require a class parameter. As class is a restricted keyword in Python, please use class_ in its place.
• In order to support the more interactive coding environments provided by Python, any call to the plot() command will immediately produce a plot. This is different from Macro, where plots are delayed until the end. The result of this is that multiple plot() commands in Python will result in multiple plot windows. Fortunately, a single plot() command can be given any number of items, including multiple pages. To see how to produce a multi-plot layout, please see the Layoutx3 Example gallery example.
• To produce multiple physical pages in a PostScript file, add calls to newpage() within the plot() command, e.g. plot(dw, data1, visdef1, newpage(), data2, visdef2).