Page History

Panel

titleColor	white
titleBGColor	#2B619E
title	Introduction

This Service will allow authorised users to retrieve and list MARS data from outside the ECMWF facilities. Users within ECMWF Member and Co-operating States may contact their Computing Representative to obtain access to MARS. All other users may request a username and password, following these instructions.

In this page you will find the step-by-step guide to install the ECMWF Web API client and create your own MARS Requests. We strongly recommend you that you get familiar with MARS.

Info

title	Information

For those used to the MARS system, it is worth noting that the client that you will install in your system is a Web API service interface that forwards the request to our systems and expects a single output file. It has no knowledge of internal ECMWF libraries such as libemos MIR or grib_apiecCodes, these operations are performed in our internal servers. Therefore, not all the functionality of the full MARS client is available, like multi-target, compute, etc...

This method is only supported on UNIX systems. It requires Python.

Panel

titleColor	white
titleBGColor	black
title	Step-by-step guide

Steps:

Info

(Optional) To follow this guide, we recommend you to have a directory in your $HOME space called "mars".

Code Block

language	bash
theme	Confluence

mkdir -p ~/mars/bin
mkdir -p ~/mars/lib

Anchor

	downloadmars
	downloadmars

Panel

titleColor	white
titleBGColor	#2B619E
title	Download Mars

Download the mars python script: mars. You have to give execution permission to the file and put it in a location in your PATH.

If the file has been downloaded to ~/mars/bin, you have to execute:

Code Block

language	bash
title	Mars

chmod +x ~/mars/bin/mars
export PATH=~/mars/bin/:$PATH

Anchorapiclientlibraryapiclientlibrary Panel

titleColor	white
titleBGColor	#2B619E
title	Install ECMWF API client library

There are 3 ways of doing this:

(recommended) Using Python pip command. This will install the necessary PYTHON libraries in the system space.

Code Block

language	bash

sudo pip install https://software.ecmwf.int/wiki/download/attachments/56664858/ecmwf-api-client-python.tgz

If you do not have access to sudo you can use the --user option of pip to install it in $HOME/.local/lib:

Code Block
pip install --user https://software.ecmwf.int/wiki/download/attachments/56664858/ecmwf-api-client-python.tgz

If you do not have access to pip:
- Extract ecmwf-api-client-python.tgz in ~/mars/lib
- Execute:

Code Block

language	bash

export PYTHONPATH=~/mars/lib:$PYTHONPATH

.

#!/usr/bin/env python
import sys
import ecmwfapi
import re
import os
import argparse

parser = argparse.ArgumentParser(description='Run MARS request.')
parser.add_argument('infile', nargs='?', default='-', type=argparse.FileType('r'),
                    help='file containing a MARS request or STDIN otherwise')

args = parser.parse_args()
req = args.infile.read()

if "WEBMARS_TARGET" in os.environ:
    target = os.environ["WEBMARS_TARGET"]
else:
    m = re.search('\\btar(g(e(t)?)?)?\s*=\s*([^\'",\s]+|"[^"]*"|\'[^\']*\')', req, re.I|re.M)
    if m is None:
        raise Exception("Cannot extract target")

    target=m.group(4)
    if target is None:
        raise Exception("Cannot extract target")

if target[0] == target[-1]:
    if target[0] in ['"', "'"]:
        target = target[1:-1]

c = ecmwfapi.ECMWFService('mars')
c.execute(req, target)

You have to give execution permission to the file and put it in a location in your PATH.

If the file has been downloaded to ~/mars/bin, you have to execute:

Code Block

language	bash
title	Mars

chmod +x ~/mars/bin/mars
export PATH=~/mars/bin/:$PATH

Anchor

	apiclientlibrary
	apiclientlibrary

Anchorkeykey

Panel

titleColor	white
titleBGColor	#2B619E
title	Install ECMWF KEY	Install ECMWF API client library

Include Page

	Install ECMWF Web API Python client library
	Install ECMWF Web API Python client library

Anchor
key
key

{ "url" : "https://api.ecmwf.int/v1", "key" : "XXXXXXXXXXXXXXXXXXXXXX", "email" : "john.smith@example.com" }

Panel

titleColor	white
titleBGColor	#2B619E
title	Install ECMWF KEY

Include Page

	Install ECMWF API Key
	Install ECMWF API Key

Login into https://apps.ecmwf.int/auth/login/ using the provided username and password.

Retrieve your key at https://api.ecmwf.int/v1/key/

Info
Note that the key expires in 1 year. You will receive an email to the registered email address 1 month before the expiration date with the renewal instructions.

Copy the information in this page and paste it in the file $HOME/.ecmwfapirc You will get something similar to

Code Block

language	bash
title	$HOME/.ecmwfapirc

Anchor
testinstallation
testinstallation

Panel

titleColor	white
titleBGColor	#2B619E
title	Test installation

Example script

Retrieval of sea surface temperature for the first 10 days of May 2002, all synoptic times. It retrieves 40 fields.

Code Block
retrieve, class = od, stream = oper, expver = 1, date = 20020501/to/20020510, time = 00/06/12/18, type = an, levtype = sfc, param = sea surface temperature, target = "sst.grib"

Save this script as test.req and execute:

Code Block

language	bash

mars test.req

The command should generate a sst.grib file containing the retrieved 40 fields in GRIB format.

Expand

title	Click to see how to retrieve directly using Python...

You can use ECMWFService Python class to execute a retrieve command:

Code Block

language	py

server = ECMWFService("mars")
server.execute({ "python":"query" }, "target")
# Alternatively the function execute can also accept a text variable with the request
server.execute(req,target)

Example:

Code Block

language	py

#!/usr/bin/env python
from ecmwfapi import ECMWFService
 
server = ECMWFService("mars")
server.execute(
    {
    "class": "od",
    "date": "20150101",
    "expver": "1",
    "levtype": "sfc",
    "param": "167.128",
    "step": "0/to/240/by/12",
    "stream": "oper",
    "time": "00",
    "type": "fc"
    },
    "target.grib")

Anchor
availability
availability

Panel

titleColor	white
titleBGColor	#2B619E
title	Check availability

To check the Data availability you should have Access to the MARS Catalogue. Please note that the MARS catalogue is a super-set of the Archive catalogue.

Info

Archive Catalogue

MARS Catalogue (restricted to authorised users)

you will view all the experiments that are available from our archive. We strongly recommend you to to navigate through MARS Catalogue to become familiar with the availability of our archive. You may select a class and start navigate trough the contents.

Please take into account some considerations about the content:

Different class and stream include different parameters, times, and steps
In each class not all the parameters are available from all steps
In each class and stream not all the steps are available from all times

The web-application above will help you to check and understand the availability. For any kind of selections the system will update the attributes in a dynamic way to reflect the current availability. (i.e. if you change the steps some parameters will be added or removed).

Info

title	Tip

We encourage users to use the " View MARS request " feature once the selection has been done.

You can also use list action with output=cost keyword to see the available data and its cost without need to retrieve it. This is useful to check the amount of data and number of tapes that your request will need to access. If your request takes longer than expected, it is a good practice to check its cost.

Expand

title	expand list action...

Code Block
list, class = od, stream = oper, expver = 1, date = 20020501/to/20020510, time = 00/06/12/18, type = an, levtype = sfc, param = sea surface temperature, output = cost, target = "sst.list"

Expand

title	List verb with Python...

Code Block

language	py

#!/usr/bin/env python

from ecmwfapi import ECMWFService
 
server = ECMWFService("mars")
req = '''
list,
  class   = od,
  stream  = oper,
  expver  = 1,
  date    = 20020501/to/20020510,
  time    = 00/06/12/18,
  type    = an,
  levtype = sfc,
  param   = sea surface temperature,
  output  = cost,
'''
server.execute(req, "sst.list")

This would produce something like:

Code Block
size=21599920; number_of_fields=40; online_size=21599920; off_line_size=0; number_of_tape_files=0; number_of_disk_files=1; number_of_online_fields=40; number_of_offline_fields=0; number_of_tapes=0;

Indicating that the size of this request will be 20.59 Mb (output is in Kb) and that it is fully located in the online disk (no need to access any tape).

Warning
Bear in mind that the maximum value of number_of_tapes recommended is 1 tape per request and should not exceed 3. Otherwise we strongly encourage you to split it in several different requests iterating by date.

If you do not set the "output=cost" keyword on the list action, the output will show you information about the type of requests and the entries:

Code Block

language	bash
title	sst.list

class   = od
expver  = 1
file[0] = hpss:/mars/prod/od/o/oper/an/sfc/marsodoper/1/an/20020501/sfc/126665.20031108.113424
id      = 126665
levtype = sfc
month   = 200205
stream  = oper
type    = an
year    = 2002
date       file length missing offset     param  time     
2002-05-01 0    539998 .       4729126    34.128 00:00:00 
2002-05-01 0    539998 .       37233582   34.128 06:00:00 
2002-05-01 0    539998 .       69738038   34.128 12:00:00 
2002-05-01 0    539998 .       102242494  34.128 18:00:00 
2002-05-02 0    539998 .       134746950  34.128 00:00:00 
2002-05-02 0    539998 .       167251406  34.128 06:00:00 
2002-05-02 0    539998 .       199755862  34.128 12:00:00 
2002-05-02 0    539998 .       232260318  34.128 18:00:00 
2002-05-03 0    539998 .       264764774  34.128 00:00:00 
2002-05-03 0    539998 .       297269230  34.128 06:00:00 
2002-05-03 0    539998 .       329773686  34.128 12:00:00 
2002-05-03 0    539998 .       362278142  34.128 18:00:00 
2002-05-04 0    539998 .       394782598  34.128 00:00:00 
2002-05-04 0    539998 .       427287054  34.128 06:00:00 
2002-05-04 0    539998 .       459791510  34.128 12:00:00 
2002-05-04 0    539998 .       492295966  34.128 18:00:00 
2002-05-05 0    539998 .       524800422  34.128 00:00:00 
2002-05-05 0    539998 .       557304878  34.128 06:00:00 
2002-05-05 0    539998 .       589809334  34.128 12:00:00 
2002-05-05 0    539998 .       622313790  34.128 18:00:00 
2002-05-06 0    539998 .       654818246  34.128 00:00:00 
2002-05-06 0    539998 .       687322702  34.128 06:00:00 
2002-05-06 0    539998 .       719827158  34.128 12:00:00 
2002-05-06 0    539998 .       752331614  34.128 18:00:00 
2002-05-07 0    539998 .       784836070  34.128 00:00:00 
2002-05-07 0    539998 .       817340526  34.128 06:00:00 
2002-05-07 0    539998 .       849844982  34.128 12:00:00 
2002-05-07 0    539998 .       882349438  34.128 18:00:00 
2002-05-08 0    539998 .       914853894  34.128 00:00:00 
2002-05-08 0    539998 .       947358350  34.128 06:00:00 
2002-05-08 0    539998 .       979862806  34.128 12:00:00 
2002-05-08 0    539998 .       1012367262 34.128 18:00:00 
2002-05-09 0    539998 .       1044871718 34.128 00:00:00 
2002-05-09 0    539998 .       1077376174 34.128 06:00:00 
2002-05-09 0    539998 .       1109880630 34.128 12:00:00 
2002-05-09 0    539998 .       1142385086 34.128 18:00:00 
2002-05-10 0    539998 .       1174889542 34.128 00:00:00 
2002-05-10 0    539998 .       1207393998 34.128 06:00:00 
2002-05-10 0    539998 .       1239898454 34.128 12:00:00 
2002-05-10 0    539998 .       1272402910 34.128 18:00:00 

Grand Total:
============

Entries       : 40
Total         : 21,599,920 (20.5993 Mbytes)

Spend some time to understand how it works, make some selections try to execute the MARS scripts.

Anchor
examples
examples

Panel

bgColor	white
titleColor	white
titleBGColor	#2B619E
title	MARS examples

Expand

title	Click here to see examples...

Section

Column

Panel

titleColor	white
titleBGColor	#2B619E
title	40 year re-analysis

Retrieval of snow depth from the ERA-40 archive for November 1993, for all analysis base times. It retrieves 120 fields. See in MARS Catalogue

Code Block

language	java

retrieve,
  class   = e4,
  stream  = oper,
  expver  = 1,
  date    = 19931101/to/19931130,
  time    = 00/06/12/18,
  type    = an,
  levtype = sfc,
  param   = sd,
  repres  = gg,
  target  = "era40.199311.sd"

Panel

titleColor	white
titleBGColor	#2B619E
title	Operational Analysis

Retrieval of sea surface temperature for the first 10 days of May 2002, all synoptic times. It retrieves 40 fields. See in MARS Catalogue

Code Block

language	java

retrieve,
  class   = od,
  stream  = oper,
  expver  = 1,
  date    = 20020501/to/20020510,
  time    = 00/06/12/18,
  type    = an,
  levtype = sfc,
  param   = sea surface temperature,
  target  = "sst"

Column

Panel

titleColor	white
titleBGColor	#2B619E
title	Ensemble Forecast

Retrieval of surface temperature an 10m wind components (U and V), 20 first members of the EPS for 2nd January 2001 for time-steps 12, 36 and 60. It retrieves 180 fields. See in MARS Catalogue

Code Block

language	java

retrieve,
  class   = od,
  stream  = enfo,
  expver  = 1,
  date    = 20010102,
  time    = 12,
  step    = 12/36/60,
  type    = pf,
  levtype = sfc,
  param   = st/10u/10v,
  number  = 1/to/20,
  target  = "perturbed.sfc"

Panel

titleColor	white
titleBGColor	#2B619E
title	Snow depth 1 month

Retrieval of snow depth from the ERA-Interim archive for December 2007, for all analysis base times. It retrieves 124 fields. See in MARS Catalogue

Code Block

language	java

retrieve,
class   = ei,
stream  = oper,
expver  = 1,
date    = 20071201/to/20071231
time    = 00/06/12/18,
type    = an,
levtype = sfc,
param   = sd,
target  = "era-int.200712.sd"

Hide content

Click here to see Python examples...

white#2B619EPython retrieve

Retrieve 1000,850,700 and 500 hPa temperatures from the 15th of February 1999 12 UTC Analysis

py

You can find more advanced examples here: MARS example requests

Info

We strongly suggest users to split their requests by date e.g. one month at a time.

If you want to get big chunks of data, please have a look to the Retrieval efficiency page.

Anchor
requestsyntax
requestsyntax

Panel

titleColor	white
titleBGColor	#2B619E
title	Brief MARS request syntax

See Brief MARS request syntax page.

Anchor
documentation
documentation

Panel

titleColor	white
titleBGColor	#2B619E
title	Documentation

Training material (slides, pdf): MARS introduction and basic concepts

Mars manual

MARS FAQ

Web - API FAQ

Space shortcuts

Page tree

Versions Compared

Old Version 42

New Version Current

Key

Anchor
key
key

Anchor
testinstallation
testinstallation

Anchor
availability
availability

Anchor
examples
examples

Anchor
requestsyntax
requestsyntax

Anchor
documentation
documentation

Related articles

Space shortcuts

Page tree

Page History

Versions Compared

Old Version 42

New Version Current

Key

Anchorkeykey

Anchortestinstallationtestinstallation

Anchoravailabilityavailability

Anchorexamplesexamples

Anchorrequestsyntaxrequestsyntax

Anchordocumentationdocumentation

Related articles

Anchor
key
key

Anchor
testinstallation
testinstallation

Anchor
availability
availability

Anchor
examples
examples

Anchor
requestsyntax
requestsyntax

Anchor
documentation
documentation