...

Use MARS list to find out the size

Using You can use the MARS catalogue and the "View MARS request" functionality  to get a list request of all the data you want can be created which can be used to find out the size and distribution of the raw data in the archive. Note the LIST verb and MARS request and modify the keywords with the information that you need. Once you have the request prepared, you should replace RETRIEVE by LIST and add the OUTPUT = COST keyword. Any post-processing keyword will be ignored by the LIST verb.

LIST,
    OUTPUT     = COST,
    CLASS      = OD,
    TYPE       = FC,
    STREAM     = OPER,
    EXPVER     = 0001,
    LEVTYPE    = ML,
    LEVELIST   = 127/128/129/130/131/132/133/134/135/136/137,
    PARAM      = 130/131/132,
    DATE       = 20170401/20170402/20170403/20170404/20170405/20170406/20170407/20170408/20170409/20170410/20170411/20170412/20170413/20170414/20170415/20170416/20170417/20170418/20170419/20170420/20170421/20170422/20170423/20170424/20170425/20170426/20170427/20170428/20170429/20170430,
    TIME       = 0000/1200,
    STEP       = 0/to/90/by/1,
    TARGET     = list.txt

When we run the list action, the following output is written to the file specified with the TARGET keyword (in this example, 'list.txt'):.

size=591270399720;
number_of_fields=180180;
online_size=206846193282;
off_line_size=384424206438;
number_of_tape_files=60;
number_of_disk_files=546;
number_of_online_fields=63033;
number_of_offline_fields=117147;
number_of_tapes=8;

Information to interpret the output:

• The size is in bytes
• online_size is the number of bytes that are in the MARS disk
• offline_size is the number of bytes that are in the MARS tapes
• number_of_tapes tells how the data is split in tapes at this particular time. This may change in the future if for some reason there is more data in the MARS disk

Use the MARS catalogue to find out how the data are distributed in files and tapes

...

For this particular case, a different "Parameter", "Level" and "Step" can be selected for a specific date and time. All the fields you can choose in this page are should be stored on the same tape file. You should get as much data as possible from this page in a single MARS retrieval request.

If we run the You can use the "Estimated download size" feature or you can run a list of the sub section of data that we you need and appears appearing in the final stage page we get:

LIST,
    CLASS      = OD,
    TYPE       = FC,
    STREAM     = OPER,
    EXPVER     = 0001,
    LEVTYPE    = ML,
    LEVELIST   = 127/128/129/130/131/132/133/134/135/136/137,
    PARAM      = 130/131/132,
    DATE       = 20170401,
    TIME       = 0000,
    STEP       = 0/to/90/by/1,
    OUTPUT     = cost,
    TARGET     = list2.txt

...

Now that we know the size of the data that we want to retrieve in one go from the same hypercube block, we can start to study how the best way to iterate to get the data for all the dates and times requiredyou need.

Split the request in sensible chunks iterating through the correct keywords

...