Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

UNDER CONSTRUCTION

What is the objective of this page?

Info

To help users to improve ERA interim MARS requests performance via the WebAPI.

(lightbulb) A good understanding of the MARS efficiency issues is essential especially for users that are interested in downloading large amounts of data.

How is the ERA interim data organised in MARS?

Info

In general it is organised, as a huge tree, with the indentation below, showing different levels down that tree:

  • type of data (analysis, forecast, 4D analysis increments)
    • year
      • month
        • type of level (model level, pressure level, surface) 
          • dates, times, steps, levels, parameters (same tape file)

What would be the natural way to group requests?

Info

The idea is to request as much data as possible from the same tape file. The natural way to group requests would be:
all parameters, all levels, all time-steps, all dates of a month

(warning) Note the following:

  1. 'all' means 'all' that the user wants. It doesn't have to be all parameters.
  2. If a user is interested only on z500,  he may request more months in one go, since the overall request will not be so big.

An ERA interim request

...

(smile) You can use the variable target to organise the requested data in files as you wish.

Code Block
languagepy
#!/usr/bin/env python
import calendar
from ecmwfapi import ECMWFDataServer
server = ECMWFDataServer()

def retrieve_interim():
    """       
       A function to demonstrate how to iterate efficiently over all days and months,
       for years 2014 and 2015 for a particular interim_request. 
       You can extend the number of years to adapt the iteration to your needs.
       You can also use the variable 'target' to organise the requested data in files as you wish.
       In the example below the data are organised in files per day. (eg '2015-12-01.grb').
    """
    yearStart = 2014
    yearEnd = 2015
    monthStart = 1
    monthEnd = 12
    for year in list(range(yearStart, yearEnd + 1)):
        for month in list(range(monthStart, monthEnd + 1)):
            numberOfDays = calendar.monthrange(year, month)[1]
            for day in list(range(numberOfDays)):
                requestDate = '%04d%02d%02d' % (
                    year, month, day + 1)
                target = "%s.grb" % (requestDate)
                interim_request(requestDate, target)

def interim_request(requestDate, target):
    """       
        An ERA interim request for analysis, pressure level data.
        You can change the keywords below to adapt it to your needs.
        (eg add or remove  levels, parameters, times etc)
    """
    server.retrieve({
        "class": "ei",
        "stream": "oper",
        "type": "an",
        "dataset": "interim",
        "date": requestDate,
        "expver": "1",
        "levtype": "pl",
        "levelist": "100/200/225/250/300/350/400/450/500/700/750/850/925/1000",
        "param": "129.128/130.128/131.128/132.128",
        "target": target,
        "time": "00/06/12/18",
        "grid": "0.75/0.75"
    })

if __name__ == '__main__':
    retrieve_interim()

Useful links

Info