Background
You need to download the appropriate software from
https://pypi.org/project/ecmwf-api-client/
in order to use these python scripts.
We need to retrieve three types of files efficiently for the two reanalyses with experiment id = 21 and experiment id = 22. The three types of retrieval all have class=ea, but have the following "stream" and "type" combinations:
- Atmospheric daily analyses (stream=oper, type=an)
- Monthly means of Daily Means (stream=moda, type=an)
- Observation departures for the data used in the reanalyses (stream=oper, type=mfb)
We provide three examples below.
Retrieval of daily "stream=oper, type=an" files
- Note that the availability of the oper, an files for rom_saf_21, experiment = 21, is given here.
- For example, look at the availability of the pressure level data for May 2011 here.
- Click on "Estimate download size". The month of pressure level data is on one tape, and has size 7 GB.
- The availability of oper, an for rom_saf_22, experiment = 22, is here.
The example below retrieves geopotential and temperature on a set of pressure levels for the rom_saf_21 dataset (experiment id = 21). This example can be modified easily for rom_saf_22 (experiment id = 22). This request is for data from 20140101 to 20151231. For efficiency reasons, the retrieval is done one calendar month at a time, so we produce 24 monthly files. These monthly files can be combined with the "cat" command if required by the user. E.g., combining three months of data is simply,
cat month1 month2 month3 > three_months.
In this example, the files are given in netcdf format. The data can also be retrieved as grib.
#!/usr/bin/env python
import calendar
from ecmwfapi import ECMWFDataServer
server = ECMWFDataServer()
def retrieve_rom_saf():
"""
A function to demonstrate how to iterate efficiently over several years and months etc
for a particular rom_saf request.
Change the variables below to adapt the iteration to your needs.
You can use the variable 'target' to organise the requested data in files as you wish.
In the example below the data are organised in files per month. (eg "rom_saf_21_daily_201510.nc")
"""
yearStart = 2014
yearEnd = 2015
monthStart = 1
monthEnd = 12
for year in list(range(yearStart, yearEnd + 1)):
for month in list(range(monthStart, monthEnd + 1)):
startDate = '%04d%02d%02d' % (year, month, 1)
numberOfDays = calendar.monthrange(year, month)[1]
lastDate = '%04d%02d%02d' % (year, month, numberOfDays)
target = "rom_saf_21_daily_%04d%02d.nc" % (year, month) # or name rom_saf_22_ ...
requestDates = (startDate + "/TO/" + lastDate)
rom_saf_request(requestDates, target)
def rom_saf_request(requestDates, target):
"""
An ROM SAF request for analysis pressure level data.
Change the keywords below to adapt it to your needs.
(eg to add or to remove levels, parameters, times etc)
Request cost per month is 3472 fields, ~200 Mbytes
"""
server.retrieve({
"class": "ea", # do not change
"dataset": "rom_saf_21", # or "rom_saf_22"
"expver": "21", # or "22"
"stream": "oper", # do not change
"type": "an", # do not change, only the analysis is available
"date": requestDates, # dates, set automatically from above
"levtype": "pl", # pressure level data (versus surface, sfc, and model level, ml)
"levelist": "100/500/700/750/850/925/1000", # levels, required only with levtype:pl and levtype:ml
"param": "129.128/130.128", # Geopotential height, Temperature (T), see http://apps.ecmwf.int/codes/grib/param-db
"target": target, # output file name, set automatically from above
"time": "00/03/06/09/12/15/18/21", # times of analysis
"grid": "1.5/1.5", # Optional for GRIB, required for NetCDF. The horizontal resolution in decimal degrees. If not set, the archived grid as specified in the data documentation is used.
"format": "netcdf", # Optional. Output in NetCDF format. Requires that you also specify 'grid'. If not set, data is delivered in GRIB format, as archived.
})
if __name__ == '__main__':
retrieve_rom_saf()
Retrieval of "stream=moda, type=an" files
The request below will retrieve all the available moda files from 200702 to 201512. This particular example is retrieving the 70 hPa zonal wind from rom_saf_22. The retrieval is grouped into decades, and rom_saf_moda_2000 contains the information from 200702 to 200912, and rom_saf-moda_2010 contains the information from 201001 to 201512. The retrieved files are given in grib format.
!/usr/bin/env python
import calendar
from ecmwfapi import ECMWFDataServer
server = ECMWFDataServer()
def retrieve_romsaf_data():
'''
A function to demonstrate how to retrieve rom_saf monthly mean data.
Change the variables below to adapt to your needs.
ROM SAF monthly data is timestamped to the first of the month, hence dates
have to be specified as a listin this format:
'20070201/20070301/20070401/.../20151101/20151201'.
Note that 20070101 is not available
Data is stored on one tape per decade, so for efficiency we split the date range into
decades, hence we get multiple date lists by decade:
'20070201/20070201/20000301/.../20051101/20051201'
In the example below the output data are organised as one file per decade:
'romsaf_moda_2010'
Please note that only decades 2000-2010 are available. The First available moda is 20070201!
'''
yearStart = 2007 # adjust to your requirements - only 2007-02-01 to 2015-12-01 is available
yearEnd = 2015 # adjust to your requirements
months = [1,2,3,4,5,6,7,8,9,10,11,12] # adjust to your requirements
months_2007 = [2,3,4,5,6,7,8,9,10,11,12] # specific for 2007 because 200701 not computed
years = range(yearStart, yearEnd+1)
print 'Years: ',years
decades = list(set([divmod(i, 10)[0] for i in years]))
decades = [x * 10 for x in decades]
decades.sort()
print 'Decades:', decades
# loop through decades and create a month list
for d in decades:
requestDates=''
for y in years:
if ((divmod(y,10)[0])*10) == d:
if y == 2007:
for m in months_2007:
requestDates = requestDates+str(y)+(str(m)).zfill(2)+'01/'
else:
for m in months:
requestDates = requestDates+str(y)+(str(m)).zfill(2)+'01/'
requestDates = requestDates[:-1]
print 'Requesting dates: ', requestDates
target = 'romsaf_moda_22_%d.grb'% (d) # specifies the output file name
print 'Output file: ', target
romsaf_request(requestDates, d, target)
# the actual data request
def romsaf_request(requestDates, decade, target):
'''
Change the keywords below to adapt to your needs.
The easiest way to do this is:
1. go to https://apps.ecmwf.int/mars-catalogue/?class=ea&stream=moda&expver=22
2. click through to the parameters you want, for any date
3. click 'View the MARS request'
'''
server.retrieve({
'class': 'ea', # do not change
'dataset': 'rom_saf_22', # or rom_saf_21
'expver': '22', # or 21
'stream': 'moda', # monthly means of daily means
'type': 'an', # analysis (versus forecast, fc)
'levtype': 'pl', # pressure data (versus pressure level, pl, and model level, ml)
'levelist': '70', # levels, required only with levtype:pl and levtype:ml
'param': '131', # zonal wind
"grid": "1.5/1.5", # Optional for GRIB, required for NetCDF. The horizontal resolution in decimal degrees. If not set, the archived grid as specified in the data documentation is used.
'date': requestDates, # dates, set automatically from above
'decade': decade, # decade set automatically from above
'target': target # output file name, set automatically from above
})
if __name__ == '__main__':
retrieve_romsaf_data()
Please note that the moda files for 200701 are not available.
Retrieval of "stream=oper, type=mfb" files
The mfb files contain information about the observations used in the reanalyses, for example the forecast and analysis departure statistics, assumed observation error statistics, etc. This data is available at the hours 00/12 each day. The example below is for rom_saf_21, and we retrieve all the GPS-RO data for the year 2014. This retrieval produces one file per month. As above, the user can then combine these monthly files if required, e.g. cat month1 month2 > both_months.
#!/usr/bin/env python
import calendar
from ecmwfapi import ECMWFDataServer
server = ECMWFDataServer()
def retrieve_rom_saf_mfb():
"""
A function to demonstrate how to iterate efficiently over several years and months etc
for a particular rom_saf mfb request.
Change the variables below to adapt the iteration to your needs.
You can use the variable 'target' to organise the requested data in files as you wish.
In the example below the data are organised in files per month. (eg "rom_saf_mfb_21_201410.odb")
Each monthly file is ~1.2 Gb for GPSRO data used exp id = 21.
"""
yearStart = 2014
yearEnd = 2014
monthStart = 1
monthEnd = 12
for year in list(range(yearStart, yearEnd + 1)):
for month in list(range(monthStart, monthEnd + 1)):
startDate = '%04d%02d%02d' % (year, month, 1)
numberOfDays = calendar.monthrange(year, month)[1]
lastDate = '%04d%02d%02d' % (year, month, numberOfDays)
target = "rom_saf_mfb_21_%04d%02d.odb" % (year, month)
requestDates = (startDate + "/TO/" + lastDate)
rom_saf_request(requestDates, target)
def rom_saf_request(requestDates, target):
"""
An ROM SAF request for a month of MFB observation data.
Change the keywords below to adapt it to your needs.
(eg to add or to remove levels, parameters, times etc)
"""
server.retrieve({
"class" : "ea", # do not change
"dataset" : "rom_saf_21", # or "rom_saf_22"
"expver" : "21", # or "22"
"stream" : "oper", # do not change
"type" : "mfb", # do not change - only mfb's available for 21 and 22
"date" : requestDates, # dates, set automatically from above
'obsgroup' : '8', # GPSRO - for other groups see https://apps.ecmwf.int/odbgov/group/
"target" : target, # output file name, set automatically from above
"time" : "00/12", # times of analysis (00 or 12 or both)
"format" : "odb", # or ascii format
})
if __name__ == '__main__':
retrieve_rom_saf_mfb()