...
| Warning |
|---|
The data is too large: 4.48 TB and split across 32 tapes. |
Use the MARS catalogue to know the data distribution in files and tapes
...
| Tip |
|---|
In this case the raw data (without post-processing) is ~144GB, which would be reduced if you interpolate to lat/lon and/or filter the area. The data is split in 3 different tapes, which is sensible. |
Now that we know the size of the data that we want to retrive in one go from the same hypercube we can start the study of the way to iterate to get the full month.
Split the request in sensible chunks iterating through the correct keywords
Using the browser we know how the data is distributed in the MARS tree. Now we focus on the current selection section. In this case:
From the top to the bottom we have to start iterating from the inner loop to the outer loop "Time", "date", ...
This is an example BASH script to loop 1 time/ 1 day in one go for one month
| Code Block | ||
|---|---|---|
| ||
#!/bin/bash PARAMS="134.128/151.128/165.128/166.128/246.228/247.228" TIMES="0000" YEAR="2017" MONTH="03" #date loop for y in ${YEAR}; do for m in ${MONTH}; do #get the number of days for this particular month/year days_per_month=$(cal ${m} ${y} | awk 'NF {DAYS = $NF}; END {print DAYS}') for my_date in $(seq -w 1 ${days_per_month}); do my_date=${YEAR}${MONTH}${my_date} #time lop for my_time in ${TIMES}; do cat << EOF > my_request_${my_date}_${my_time}.mars REQUEST, CLASS = OD, TYPE = PF, STREAM = ENFO, EXPVER = 0001, LEVTYPE = SFC, PARAM = ${PARAMS}, DATE = ${my_date}, TIME = ${my_time}, STEP = 0/1/2/3/4/5/6/7/8/9/10/11/12/13/14/15/16/17/18/19/20/21/22/23/24/25/26/27/28/29/30/31/32/33/34/35/36/37/38/39/40/41/42/43/44/45/46/47/48/49/50/51/52/53/54/55/56/57/58/59/60/61/62/63/64/65/66/67/68/69/70/71/72/73/ 74/75/76/77/78/79/80/81/82/83/84/85/86/87/88/89/90/93/96/99/102/105/108/111/114/117/120/123/126/129/132/135/138/141/144/150/156/162/168/174/180/186/192/198/204/210/216/222/228/234/240/246/252/258/264/270/276/282/288/294/300/306/3 12/318/324/330/336/342/348/354/360, NUMBER = 1/to/50, TARGET = enfo_pf_${my_date}_${my_time}.grib EOF mars my_request_${my_date}_${my_time}.mars if [ $? -eq 0 ]; then rm -f my_request_${my_date}_${my_time}.mars fi done done done done |