Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

UNDER CONSTRUCTION!!!

What is the objective of this page?

Info

To help users to improve TIGGE MARS requests performance via the WebAPI.

(lightbulb) A good understanding of the MARS efficiency issues is essential especially for users that are interested in downloading large amounts of data.

How is the TIGGE data organised in MARS?

Info

In general it is organised, as a huge tree, with the indentation below, showing different levels down that tree:

What would be the natural way to group requests?

Info

The idea is to request as much data as possible from the same tape file. The natural way to group requests would be:
all parameters, all members, all origins, all steps  for a time,  for a date, for a type of level

(warning) Note the following:

  1. 'all' means 'all' that the user wants. It doesn't have to be all parameters or all members
  2. If a user is interested only on z500,  he may request more dates in one go, since the overall request will not be so big.

Best practise to iterate over all  dates and all times for all levels (or for a particular level eg Pressure levels)

Info

(lightbulb) The best approach is to iterate over the dates and for each date iterate over the times.

(warning) At this point you may wish to  check TIGGE availability and to view a TIGGE request

Info
for date in dates
for time in times
TIGGE-request(date, time)
(Here you must add everything you need)

Web-API examples:

A TIGGE request for all the available dates

Info
  • The objective of this example is to demonstrate how to iterate efficiently over some dates (eg "2014-10-01" and "2014-11-01")  a requests  for TIGGE , "perturbed forecast",, "pressure" and "sfc" level, for ECMWF Center,
  • It can be used as a starting point, however you need to keep in mind that you have to adapt it to your needseg to set the keyword values according to your requirements ("param", "levtype", "step" etc).
  • In this way you can extend this request to download the whole TIGGE.

(thumbs up) Keep in mind that the most efficient way is to organise your requests per dates and times

ie select a set of dates and times and add,  the steps, origins, levels parameters that you need.

(thumbs down)   It is not efficient to organise your requests per parameter, origin  or  step. eg to submit a different request per parameter for a set of date

Code Block
languagepy
#!/usr/bin/env python
from ecmwfapi import ECMWFDataServer
server = ECMWFDataServer()
   
def retrieve_tigge_data():
    dates = ["2014-10-01", "2014-11-01"]
    times = ["00", "12"]
    for date in dates:
         for time in times:
             target = "ecmwf_sfc_%s_%s.grb" % (date, time)
             tigge_pf_sfc_request(date, time, target)
             target = "ecmwf_pl_%s_%s.grb" % (date, time)
             tigge_pf_pl_request(date, time, target)

def tigge_pf_pl_request(date, time, target):
    """
       A TIGGE request for perturbed forecast, pressure level, ECMWF Center.
       Please note that a subset of the available data is requested below.
       The cost of this request is xxx fields and xxx GB
       Change the keywords below to adapt it to your needs. (ie to add more parameters, or numbers etc)
    """
    server.retrieve({
        "class": "ti",
        "dataset": "tigge",
        "date": date,
        "expver": "prod",
        "grid": "0.5/0.5",
        "levelist": "200/250/300/500/",
        "levtype": "pl",
        "number": "1/TO/10",
        "origin": "ecmf",
        "param": "130/131/132/133",
        "step": "0/TO/96/BY/12",
        "target": target,
        "time": time,
        "type": "pf",
    })

def tigge_pf_sfc_request(date, time, target):
    """
       A TIGGE request for perturbed forecast, sfc, ECMWF Center.
       Please note that a subset of the available data is requested below.
       The cost of this request is xxx fields and xxx GB
       Change the keywords below to adapt it to your needs. (ie to add more parameters, or numbers etc)
    """
    server.retrieve({
         "class": "ti",
         "dataset": "tigge",
         "date": date,
         "expver": "prod",
         "grid": "0.5/0.5",
         "levtype": "sfc",
         "number": "1/TO/10",
         "origin": "ecmf",
         "param": "146/147/151/165/166/167/168/176/177",
         "step": "0/TO/96/BY/12",
         "target": target,
         "time": time,
         "type": "pf",
       })

if __name__ == '__main__':
    retrieve_tigge_data()
                                          

 

Useful links

Info