Page History

An ECMWF reforecast request for all the available hindcastDates of an ECMWF model version date

Info

The objective of this example is to demonstrate how to get efficiently all the available hincastDates for an ECMWF model version date.
To start writing your request, firstly you need to select a model version date and to find the corresponding Hindcast Dates. Please check ECMWF availability
Keep in mind that the ECMWF reforecast configuration is "on the fly". For more information read a brief description of reforecasts.
The requests below

!!!!!!!!!!!!!!!!!!!!!!!!! UNDER CONSTRUCTION !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

Table of Contents

What is the objective of this page?

Info

To help users to improve S2S ECMWF MARS requests performance via the WebAPI.

A good understanding of the MARS efficiency issues is essential especially for users that are interested in downloading large amounts of data.

How is the S2S reforecast data organised in MARS?

Info

In general it is organised, as a huge tree, with the indentation below, showing different levels down that tree:

centre (BoM, ECMWF, NCEP, JMA, ...)
- realtime or reforecast
  - type of data (control forecast or perturbed forecast)
    - type of level (single level or pressure level or potential temperature)
      - model version date (2014-01-01 or ...)
        hindcast dates (2013-09-01, 2013-09-06, 2013-09-11, 2013-09-16, 2013-09-21, 2013-09-26, ...)
        time and steps
        members (for perturbed forecast)
        levels (for pl or pt)
        parameters

What would be the natural way to group requests?

Info

The idea is to request as much data as possible from the same tape file. The natural way to group requests would be:
all parameters, all levels, all members, all time-steps for 1 hindcast date for a type of level for a type

Note the following:

'all' means 'all' that the user wants. It doesn't have to be all parameters.
If a user is interested only on z500, he may request more hindcast dates in one go, since the overall request will not be so big.

Best practise to iterate over all hindcastDates of several hindcastYears for ECMWF

Info

The best approach is to iterate over the hindcastYears. For each hindcastYear iterate over all the available hindcastMonths and for each hindcastMonth iterate over all the available hindcastDays.

At this point you may wish to check ECMWF availability and to view an ECMWF request

Info
for hindcastYear in hindcastYears for hindcastMonth in hindcastMonths for hindcastDay in hindcastDays hindcastDate = hindcastYear-hindcastMonth-hindcastDay S2S-request(hindcastDate)

Web-API examples:

An ECMWF reforecast request for all the available hindcastDates

Info

The objective of this example is to demonstrate how to iterate efficiently over all the available hindcastYears, hindcastMonths and hindcastDays for an ECMWF model version date.
It can be used as a starting point, however you need to keep in mind that you have to adapt it to your needseg to set the keyword values according to your requirements ("param", "levtype", "step" etc).
In this way you can extend this request example to download the whole S2S ECMW Freforecast. Don't forget to check ECMWF availability

Please note:

the most efficient way is to request all hindcastDates of a hindcastMonth, in one request, like the example below.

you
ECMWF reforecast.
You can use the variable target to
write .
organise the requested data in files as you wish.
In the example below the data is written per leveltype (sfc, pl) per hindcastMonth

Code Block

language	py

#!/usr/bin/env python
from ecmwfapi import ECMWFDataServer
server = ECMWFDataServer()

origin = "ecmf"

# Step 1: Select a Model Version Date
modelVersionDate = "2015-12-03"

def retrieve_ECMWF_reforecast():
    """
       A function to demonstrate how to iterate efficiently over all hindcastYears, hindcastMonths etc
       for a particular ECMWF_reforecast_request.
       Change the variables below to adapt the iteration to your needs
    """
    # ECMWF availability is every 5 days: 1, 6, 11, 16, 21, 26
    hindcastDays = # Step 2: Set the hindcast Dates you wish to request
# For the model version "2015-12-03" the available hindcast dates are listed below:
hindcastDates = ["1995-12-03/", "1996-12-03/", "1997-12-03/", "1998-12-03/", "1999-12-03/", "2000-12-03/",
                 "2001-12-03/", "2002-12-03/", "2003-12-03/", "2004-12-03/", "2005-12-03/", "2006-12-03/", "2007-12-03/",
                 "2008-12-03/", "2009-12-03/", "2010-12-03/", "2011-12-03/", "2012-12-03/", "2013-12-03/", "2014-12-03"]

    #Step 1: Iterate over all the available hindcastYear(s)
    for hindcastYear in list(range(hindcastYearStart, hindcastYearEnd + 1)def retrieve_ECMWF_reforecast():
        #Step 2: Iterate over all the available hindcastMonths(s)       """
       A forfunction hindcastMonthto in list(range(hindcastMonthStart, hindcastMonthEnd + 1)):
            hindcastDates = []
 demonstrate how to retrieve efficiently all hindcastDates
           #Step 3: Create the list of the available hindcastDates
            for hindcastDay in hindcastDays:
                hindcastDate = '%04d%02d%02d' % (for a particular ECMWF reforecast model version.
       Change the variables below to adapt the request to     hindcastYear, hindcastMonth, hindcastDay)your needs
                hindcastDates.append(hindcastDate)"""
           
    # Please note      #Please note: that the steps"sfc" 4and and"pl" 5requests below could run in parallel
    #       
            #Step 4: Get all the available perturbed forecast,Step 1: Get pressure level data
            pfplTarget = "%s_%s_%04d%02d%s.grb" % (
                origin, "pfpl", hindcastYear, hindcastMonthmodelVersionDate)
            ECMWF_reforecast_pf_pl_request("/".join(hindcastDates), pfplTarget)
           
    #        #Step 5Step 2: Get all the available perturbed forecast, surface data
            pfsfcTarget = "%s_%s_%04d%02d%s.grb" % (
                origin, "pfsfc", hindcastYear, hindcastMonthmodelVersionDate)
            ECMWF_reforecast_pf_sfc_request("/".join(hindcastDates), pfsfcTarget)

def ECMWF_reforecast_pf_pl_request(hindcastDate, target):
    """
       An AECMWF ECMWFreforecastreforecast, perturbed forecast, pressure level, request.
       TheChange costthe ofkeywords thisbelow requestto isadapt 571,392it fieldsto andyour 11needs.1352 Gbytes(eg 
to add or remove some steps or Change the keywords below to adapt it to your needs.parameters etc)
    """
    server.retrieve({
        "class": "s2",
        "dataset": "s2s",
        "date": modelVersionDate,
        "expver": "prod",
        "hdate": hindcastDatehindcastDates,
        "levtype": "pl",
        "levelist": "10/50/100/200/300/500/700/850/925/1000",
        "origin": origin,
        "param": "130/131/132/133/135/156",
        "step": "24/to/1488768/by/24",
        "stream": "enfh",
        "target": target,
        "time": "00",
        "number": "1/to2/3/324",
        "type": "pf",
    })

def ECMWF_reforecast_pf_sfc_request(hindcastDate, target):
    """
    An   A ECMWFreforecast, perturbed forecast, sfc request.
       TheChange costthe ofkeywords thisbelow requestto isadapt 383,040it fieldsto andyour 7needs.1 GB
(eg to add or remove some steps Changeor the keywords below to adapt it to your needs.parameters etc)
    """
    server.retrieve({
        "class": "s2",
        "dataset": "s2s",
        "date": modelVersionDate,
        "expver": "prod",
        "hdate": hindcastDate,
        "levtype": "sfc",
        "origin": origin,
        "param": "311/33/34/59/121/122/134/136/146/147/151/165/166/167/168/169/175/176/177/179/180/181/235/228086/228095/228096/228141/228143/228144/228164/228228",
        "step": "24/to/1488744/by/24",
        "stream": "enfh",
        "target": target,
        "time": "00",
        "number": "1/2/to3/324",
        "type": "pf",
    })

if __name__ == '__main__':
    retrieve_ECMWF_reforecast()

Useful links

Info
ECMWFreECMWF re-forecast requests ECMWF Web API Home ECMWFreECMWF re-forecast availability A brief description of reforecasts

Space shortcuts

Page tree

Versions Compared

Old Version 5

New Version Current

Key

An ECMWF reforecast request for all the available hindcastDates of an ECMWF model version date

What is the objective of this page?

How is the S2S reforecast data organised in MARS?

What would be the natural way to group requests?

Best practise to iterate over all hindcastDates of several hindcastYears for ECMWF

Web-API examples:

An ECMWF reforecast request for all the available hindcastDates