| Table of Contents |
|---|
Identify the data you want
- SFC, 6 parameters
- Time: 00
- Dates: 1 to 31 March 2019
- all steps (0/to/90/by/1, 93/to/144/by/3 and 150/to/360/by/6)
- all perturbed members (1/to/50)
- Area: Europe
- Output grid: regular lat/lon 0.5/0.5
Use MARS list to find out the size
You can use the MARS catalogue and the "View MARS request" to get a MARS request and modify the keywords with the information that you need. Once you have the request prepared, you should replace RETRIEVE by LIST and add the OUTPUT = COST keyword. Any post-processing keyword will be ignored by the LIST verb.
...
| Warning |
|---|
The data is too large: 1,348,500 fields, 19 tapes. |
Use the MARS catalogue to find out how the data are distributed in files and tapes
Using the MARS catalogue, browse to the data you want to retrieve until you reach the final stage which gives a selection and several options: http://apps.ecmwf.int/mars-catalogue/?stream=enfo&levtype=sfc&time=00%3A00%3A00&expver=1&month=mar&year=2019&date=2019-03-01&type=pf&class=od
...
| Tip |
|---|
The size of the file that will be transferred to the system can be established by downloading a single field using the Post-processing keywords (usually GRID and AREA). Then multiply the size of the file obtained containing this single field by the total number of fields that you want to retrieve in a single request: number_of_fields=43500. |
Now that we know the size of the data that we want to retrieve in one go from the same block we can start the study how best to iterate to retrieve the full month of data required.
Split the request in sensible chunks iterating through the correct keywords
The browser can now be used to find out how the data are distributed in the MARS tree. If you have a look to the "Current selection" section for this specific example shows:
...
This is an example BASH script to loop, retrieving data for one date and time at a time for a full month.
Bash script example
| Code Block | ||
|---|---|---|
| ||
#!/bin/bash
#this example will filter the area of Europe (N/W/S/E) and interpolate the final fields to a lat/lon 0.5/0.5 degrees
AREA="73.5/-27/33/45"
GRID="0.5/0.5"
# fixed selection from the same block
STEP="0/1/2/3/4/5/6/7/8/9/10/11/12/13/14/15/16/17/18/19/20/21/22/23/24/25/26/27/28/29/30/31/32/33/34/35/36/37/38/39/40/41/42/43/44/45/46/47/48/49/50/51/52/53/54/55/56/57/58/59/60/61/62/63/64/65/66/67/68/69/70/71/72/73/74/75/76/77/78/79/80/81/82/83/84/85/86/87/88/89/90/93/96/99/102/105/108/111/114/117/120/123/126/129/132/135/138/141/144/150/156/162/168/174/180/186/192/198/204/210/216/222/228/234/240/246/252/258/264/270/276/282/288/294/300/306/312/318/324/330/336/342/348/354/360,"
PARAMS="134.128/151.128/165.128/166.128/246.228/247.228"
NUMBER="1/to/50"
TIMES="0000"
YEAR="2019"
MONTH="03"
for y in ${YEAR}; do
for m in ${MONTH}; do
#get the number of days for this particular month/year
days_per_month=$(cal ${m} ${y} | awk 'NF {DAYS = $NF}; END {print DAYS}')
#date loop
for my_date in $(seq -w 1 ${days_per_month}); do
my_date=${YEAR}${m}${my_date}
#time lop
for my_time in ${TIMES}; do
cat << EOF > my_request_${my_date}_${my_time}.mars
RETRIEVE,
CLASS = OD,
TYPE = PF,
STREAM = ENFO,
EXPVER = 0001,
LEVTYPE = SFC,
GRID = ${GRID},
AREA = ${AREA},
PARAM = ${PARAMS},
DATE = ${my_date},
TIME = ${my_time},
STEP = ${STEP},
NUMBER = ${NUMBER},
TARGET = "enfo_pf_${my_date}_${my_time}.grib"
EOF
mars my_request_${my_date}_${my_time}.mars
if [ $? -eq 0 ]; then
rm -f my_request_${my_date}_${my_time}.mars
fi
done
done
done
done |
| Note |
|---|
You can use the multi-target feature in the TARGET keyword if you are using the full MARS client (from within ECMWF computing facilities). You can also use ecCodes to post-process the TARGET file afterwards. |
...