Table of Contents
*************** This page is under construction! *****************
What is the objective of this page?
Info |
---|
To help users to improve S2S CMA MARS requests performance via the WebAPI.
|
How is the S2S reforecast data organised in MARS?
Info |
---|
In general it is organised, as a huge tree, with the indentation below, showing different levels down that tree:
|
What would be the natural way to group requests?
Info |
---|
The idea is to request as much data as possible from the same tape file. The natural way to group requests would be:
|
Best practise to iterate over all hindcastDates of several hindcastYears for
...
BoM
Info |
---|
For each hindcastYear iterate over all the available hindcastMonths and for each hindcastMonth iterate over all the available hindcastDays.
|
...
Info |
---|
for hindcastYear in hindcastYears |
Web-API examples:
A
...
BoM reforecast request for
...
all the available hindcastDates
Info |
---|
|
...
|
Code Block | ||
---|---|---|
| ||
#!/usr/bin/env python from ecmwfapi import ECMWFDataServer server = ECMWFDataServer() origin = "ammc" modelVersionDate = "2014-05-01" # This is the first model version for CMA (babj) hindcastDate = "2014-04-01" # The selected hindcast date server = ECMWFDataServer() server.retrieve({ 01-01" def retrieve_BoM_reforecast(): """ A function to demonstrate how to iterate efficiently over all hindcastYears, hindcastMonths etc for a particular BoM_reforecast_request. Change the variables below to adapt the iteration to your needs """ hindcastYearStart = 1981 hindcastYearEnd = 2013 hindcastMonthStart = 1 hindcastMonthEnd = 12 # BoM availability is every 5 days: 1, 6, 11, 16, 21, 26 hindcastDays = [1, 6, 11, 16, 21, 26] #Step 1: Iterate over all the available hindcastYear(s) for hindcastYear in list(range(hindcastYearStart, hindcastYearEnd + 1)): #Step 2: Iterate over all the available hindcastMonths(s) for hindcastMonth in list(range(hindcastMonthStart, hindcastMonthEnd + 1)): hindcastDates = [] #Step 3: Create the list of the available hindcastDates for hindcastDay in hindcastDays: hindcastDate = '%04d%02d%02d' % ( hindcastYear, hindcastMonth, hindcastDay) hindcastDates.append(hindcastDate) #Please note: the steps 4 and 5 below could run in parallel #Step 4: Get all the available perturbed forecast, pressure level data pfplTarget = "%s_%s_%04d%02d.grb" % ( origin, "pfpl", hindcastYear, hindcastMonth) BoM_reforecast_pf_pl_request("/".join(hindcastDates), pfplTarget) #Step 5: Get all the available perturbed forecast, surface data pfsfcTarget = "%s_%s_%04d%02d.grb" % ( origin, "pfsfc", hindcastYear, hindcastMonth) BoM_reforecast_pf_sfc_request("/".join(hindcastDates), pfsfcTarget) def BoM_reforecast_pf_pl_request(hindcastDate, target): """ A BoM reforecast, perturbed forecast, pressure level, request. The cost of this request is 571,392 fields and 11.1352 Gbytes Change the keywords below to adapt it to your needs. """ server.retrieve({ "class": "s2", "dataset": "s2s", "date": modelVersionDate, "expver": "prod", "hdate": hindcastDate, "levtype": "sfcpl", "origin": "babj", levelist": "10/50/100/200/300/500/700/850/925/1000", "origin": origin, "param": "165/166130/131/132/133/135/156", "step": "24/to/14401488/by/24", "stream": "enfh", "target": "data.pf.sfc"target, "time": "00", "number": "1/2to/332", "type": "pf", }) |
A CMA reforecast request for several hindcastDates
...
Please note: use the variable "target" to write each hindcastDate on a separate file .
v1.0
Code Block | ||
---|---|---|
| ||
#!/usr/bin/env python from ecmwfapi import ECMWFDataServer server = ECMWFDataServer() origin = "babj" modelVersionDate = "2014-05-01" def retrieve_data(hindcastDate,modelVersionDate,centre): target = "target_s2s_%s_%s.grb" % centre % hindcastDate) def BoM_reforecast_pf_sfc_request(hindcastDate, target): """ A BoM reforecast, perturbed forecast, sfc request. The cost of this request is 383,040 fields and 7.1 GB Change the keywords below to adapt it to your needs. """ server.retrieve({ "class": "s2", "dataset": "s2s", "date": modelVersionDate, "expver": "prod", "hdate": hindcastDate, "levtype": "sfc", "origin": centreorigin, "param": "16531/34/121/122/136/146/147/151/167/168/169/175/176/177/179/180/181/235/228086/228095/228096/228141/228143/228144/228164/228228", "step": "024/to/1488/by/24", "stream": "enfh", "target": target, "time": "00", "typenumber": "cf1/to/32", }) hindcastYearList = ["2012", "2013"] hindcastMonthList = ["08", "09"] hindcastDayList = ["01type",: "02pf"] , for hindcastYear in hindcastYearList: for hindcastMonth in hindcastMonthList: for hindcastDay in hindcastDayList: hindcastDate = hindcastYear + hindcastMonth + hindcastDay retrieve_data(hindcastDate) |
Some further help for setting BoM hindcast dates
Code Block | ||||
---|---|---|---|---|
| ||||
for hindcastYear in ["2012", "2013"]: for hindcastMonth in ["08", "09"]: }) if __name__ == '__main__': retrieve_BoM_reforecast() for hindcastDay in ["01" "06" "11" "16" "21" "26"]: hindcastDate = hindcastYear+hindcastMonth+hindcastDay retrieve_data(hindcastDate,"2014-01-01","ammc") |
Useful links
Info |
---|
*****************************************************************************************************