| Table of Contents |
|---|
Identify the data you want to retrieve
- ML, 3 parameters
- Time: 00 and 12
- Dates: 1 to 30 April 2017
- first 90 steps, every hour (0/to/90/by/1)
- 10 levels closest to the surface (127/to/137)
- Area: Europe
- Output grid: regular lat/lon 0.5/0.5
Use MARS "list" verb to find out the size of your request
You can use the MARS catalogue and the "View MARS request" to get a MARS request and modify the keywords with the information that you need. Once you have the request prepared, you should replace RETRIEVE by LIST and add the OUTPUT = COST keyword. Any post-processing keyword will be ignored by the LIST verb.
| No Format |
|---|
LIST,
OUTPUT = COST,
CLASS = OD,
TYPE = FC,
STREAM = OPER,
EXPVER = 0001,
LEVTYPE = ML,
LEVELIST = 127/128/129/130/131/132/133/134/135/136/137,
PARAM = 130/131/132,
DATE = 20170401/20170402/20170403/20170404/20170405/20170406/20170407/20170408/20170409/20170410/20170411/20170412/20170413/20170414/20170415/20170416/20170417/20170418/20170419/20170420/20170421/20170422/20170423/20170424/20170425/20170426/20170427/20170428/20170429/20170430,
TIME = 0000/1200,
STEP = 0/to/90/by/1,
TARGET = list.txt |
When we run the list actionverb, the following output is written to into the file specified with the TARGET keyword (in this example, 'list.txt').
| No Format |
|---|
size=591270399720; number_of_fields=180180; online_size=206846193282; off_line_size=384424206438; number_of_tape_files=60; number_of_disk_files=546; number_of_online_fields=63033; number_of_offline_fields=117147; number_of_tapes=8; |
Information on how to interpret the this output:
- The size is in bytes
- online_size is the number of bytes that are in the MARS disk
- offline_size is the number of bytes that are in the MARS tapes
- number_of_tapes tells how the off_line data is split in tapes at this particular time. This may change in the future if for some reason there is more data in the MARS disk
| Warning |
|---|
The This request is too large: 180,000 fields, 8 tapes. |
Use the MARS catalogue to find out how the data are distributed in files and tapes
...
| Tip |
|---|
You should get as much data as possible from this page in a single MARS retrieval request. |
| Warning |
|---|
You should not iterate through any of this keywords. |
You can use the "Estimated download size" feature or you can run a with the list verb of the sub section of data that you need appearing in the this final stage page:
| No Format |
|---|
LIST,
CLASS = OD,
TYPE = FC,
STREAM = OPER,
EXPVER = 0001,
LEVTYPE = ML,
LEVELIST = 127/128/129/130/131/132/133/134/135/136/137,
PARAM = 130/131/132,
DATE = 20170401,
TIME = 0000,
STEP = 0/to/90/by/1,
OUTPUT = cost,
TARGET = list2.txt |
...
| Tip |
|---|
In this case the raw data (without post-processing) is ~9.85GB. The total size of the data to be transferred to your system will be less if you interpolate to lat/lon and/or filter the area. The data is stored on one single tape, which is sensible and an efficient way to retrieve it. |
| Tip |
|---|
The size of the file that will be transferred to the system can be established by downloading a single field using the Post-processing keywords (usually GRID and AREA). Then multiply the size of the file obtained containing this single field by the total number of fields that you want to retrieve in a single request: . In this case, number_of_fields=3003. |
Now that we know the size of the data that we want to retrieve in one go from the same block, we can study the best way to iterate to get the data you need.
...