...
- SFC, 6 parameters
- Time: 00, March 2017
- all steps (0/to/90/by/1, 93/to/144/by/3 and 150/to/360/by/6)
- all numbers (1/to/50)
- Area: Europe
- Output grid: regular lat/lon 0.5/0.5
Use MARS list to find out the size
Using the MARS catalogue and the "View MARS request" functionality you can create a list request of all the data you want which would tell you the size and distribution of the raw data in the archive. Note the LIST verb and the OUTPUT = COST keyword.
| Code Block |
|---|
LIST,
CLASS = OD,
TYPE = PF,
STREAM = ENFO,
EXPVER = 0001,
LEVTYPE = SFC,
PARAM = 134.128/151.128/165.128/166.128/246.228/247.228,
DATE = 20170301/to/20170331,
TIME = 0000,
STEP = 0/1/2/3/4/5/6/7/8/9/10/11/12/13/14/15/16/17/18/19/20/21/22/23/24/25/26/27/28/29/30/31/32/33/34/35/36/37/38/39/40/41/42/43/44/45/46/47/48/49/50/51/52/53/54/55/56/57/58/59/60/61/62/63/64/65/66/67/68/69/70/71/72/73/74/75/76/77/78/79/80/81/82/83/84/85/86/87/88/89/90/93/96/99/102/105/108/111/114/117/120/123/126/129/132/135/138/141/144/150/156/162/168/174/180/186/192/198/204/210/216/222/228/234/240/246/252/258/264/270/276/282/288/294/300/306/312/318/324/330/336/342/348/354/360,
NUMBER = 1/to/50,
OUTPUT = cost,
TARGET = list.txt |
...
Using the MARS catalogue we browse the data we want to get until we get to the final stage which gives us a selection and several options: http://apps.ecmwf.int/mars-catalogue/?stream=enfo&levtype=sfc&time=00%3A00%3A00&expver=1&month=mar&year=2017&date=2017-03-01&type=pf&class=od
For this particular case we can choose different "Parameter", "Number" and "Step". This means that all the fields you can choose in this page are in the same tape file. Therefore users should get as much data as possible from this page.
...
| Tip |
|---|
In this case the raw data (without post-processing) is ~144GB, which would be reduced if you interpolate to lat/lon and/or filter the area. The data is split in 3 different tapes, which is sensible. |
| Tip |
|---|
To know the size of the file that will be transferred to the system, you can download 1 field using the Post-processing keywords (usually GRID and AREA). Then you should multiply the size of the 1 field file by the number of fields that you want to retrieve in 1 request: number_of_fields=43500. |
Now that we know the size of the data that we want to retrive in one go from the same hypercube we can start the study of the way to iterate to get the full month.
...
| Code Block | ||
|---|---|---|
| ||
#!/bin/bash #this example will filter the area of Europe (N/W/S/E) and interpolate the final fields to a lat/lon 0.5/0.5 degrees AREA="73.5/-27/33/45" GRID="0.5/0.5" PARAMS="134.128/151.128/165.128/166.128/246.228/247.228" TIMES="0000" YEAR="2017" MONTH="03" #date loop for y in ${YEAR}; do for m in ${MONTH}; do #get the number of days for this particular month/year days_per_month=$(cal ${m} ${y} | awk 'NF {DAYS = $NF}; END {print DAYS}') for my_date in $(seq -w 1 ${days_per_month}); do my_date=${YEAR}${MONTH}${my_date} #time lop for my_time in ${TIMES}; do cat << EOF > my_request_${my_date}_${my_time}.mars REQUEST, CLASS = OD, TYPE = PF, STREAM = ENFO, EXPVER = 0001, LEVTYPE = SFC, GRID = ${GRID}, AREA = ${AREA}, PARAM = ${PARAMS}, DATE = ${my_date}, TIME = ${my_time}, STEP = 0/1/2/3/4/5/6/7/8/9/10/11/12/13/14/15/16/17/18/19/20/21/22/23/24/25/26/27/28/29/30/31/32/33/34/35/36/37/38/39/40/41/42/43/44/45/46/47/48/49/50/51/52/53/54/55/56/57/58/59/60/61/62/63/64/65/66/67/68/69/70/71/72/73/74/75/76/77/78/79/80/81/82/83/84/85/86/87/88/89/90/93/96/99/102/105/108/111/114/117/120/123/126/129/132/135/138/141/144/150/156/162/168/174/180/186/192/198/204/210/216/222/228/234/240/246/252/258/264/270/276/282/288/294/300/306/312/318/324/330/336/342/348/354/360, NUMBER = 1/to/50, TARGET = enfo_pf_${my_date}_${my_time}.grib EOF mars my_request_${my_date}_${my_time}.mars if [ $? -eq 0 ]; then rm -f my_request_${my_date}_${my_time}.mars fi done done done done |
...