...

ssh ecs-login
cd $SCRATCH

On login node one can find ECS/HPC "mars" command which is installed as part of MARS client and can be executed called without loading any module. The solutions below provide valid MARS requests which can be passed to the mars command as:

mars < myrequestfile
                            # or
mars myrequestfile

Basic MARS usage

Simple retrieves

1)

Retrieve 1000,850,700 and 500 hPa temperatures from the 15th of October 2023 12 UTC Analysis  > see in MARS Catalogue

...

ls t.grb

This morning you have learned how to use ecCodes. You can use it now to check content of your GRIB file:

module load ecmwf-toolbox
grib_ls t.grb

How many messages messages does the GRIB file containscontain? Does it match to the number of expected fields retrieved from MARS?

2)

We run the firs MARS request on a login node of ECS. Because of the limitations imposed on interactive sessions (e.g., memory, cpu time) it is better to use batch jobs to run MARS request. Yesterday, you learned how to run fractional jobs. Let's use it know to execute next MARS request. The following template script can be used for this purpose:

#!/bin/bash
  
#SBATCH --job-name=mars_job
#SBATCH --qos=ef
#SBATCH --time=1:00:00
#SBATCH --output=mars.%j.out
#SBATCH --error=mars.%j.out

set -xe

# Enable multi target option!
export MARS_MULTITARGET_STRICT_FORMAT=1 

# to run in debug mode
#export MARS_DEBUG=1

export SCRATCH
export PERM

mars <<EOF
#{paste your MARS request here}
EOF

Use the template above to retrieve wind components valid for 5 days ago at 12:00 and 18:00 from forecast made 7 days ago at 12 UTC at all 137 model levels  > see in MARS Catalogue

retrieve,
    class    = od,
    stream   = oper,
    type     = forecast,
    expver   = 1,
    param    = 131/132, 
    levtype  = model level,
    levelist = 1/to/137,
    date     = -7,
    time     = 12,
    step     = 48/54, 
    target   = "$SCRATCH/wind.grb"

Submit your MARS job with:

ecsbatch mars_job.sh

You can check the job status with:

watch ecsqueue -u $USER

once the MARS job is complete, please check the job's output file, read MARS messages, check for any ERROR or WARNING messages. Which step has taken most of the request time:

From now on, we will use mars_job.sh template to run all MARS requests are batch jobs.

3)

Use the same batch job template (mars_job.sh) from example 2) to run MARS request retrieving the 1000 hPa geopotential from two weeks old 12 UTC forecast (> see in MARS Catalogue) steps: 12 hour to 240 hour with 12 hour intervals:

2)

We run the first MARS request on a login node of ECS. Because of the limitations imposed on interactive sessions (e.g., memory, cpu time) it is better to use batch jobs to run MARS request which can be large with long runtime. Yesterday, you learned how to run fractional jobs. Let's use it know to execute next MARS request. The following template script can be used for this purpose:

#!/bin/bash
  
#SBATCH --job-name=mars_job
#SBATCH --qos=ef
#SBATCH --time=1:00:00
#SBATCH --output=mars.%j.out
#SBATCH --error=mars.%j.out

# For larger requests downloading data from tapes
# extend walltime "--time" to 1 day or longer 

set -xe

# Enable multi target option:
export MARS_MULTITARGET_STRICT_FORMAT=1 

export SCRATCH
export PERM

mars <<EOF
#{paste your MARS request here}
EOF

More examples can be found on: HPC2020: example Slurm serial batch job scripts for ECS

Use the template above to retrieve wind components valid for 5 days ago at 12:00 and 18:00 from forecast made 7 days ago at 12 UTC at all 137 model levels  > see in MARS Catalogue

retrieve,
    class    = od,
    stream   = oper,
    type     = forecast,
    expver   = 1,
    param    = 131/132, 
    levtype  = model level,
    levelist = 1/to/137,
    date     = -7,
    time     = 12,
    step     = 48/54, 
    target   = "$SCRATCH/wind.grb"

Submit your MARS job with:

ecsbatch mars_job.sh

You can check the job status with:

watch ecsqueue -u $USER

once the MARS job is complete, please check the job's output file, read MARS messages, check for any ERROR or WARNING messages. Read "INFO" messages, which action has taken longest time:

From now on, we will use mars_job.sh template to run all MARS requests as batch jobs.

3)

Use the same batch job template (mars_job.sh) from example 2) to run MARS request retrieving the 1000 hPa geopotential from two weeks old 12 UTC forecast (> see in MARS Catalogue) steps: 12 hour to 240 hour with 12 hour intervals:

retrieve,
    class = od,
    stream = oper,
    type = fc,
    expver = 1,
    date = -14,
    time = 12,
    step = 12/to/240/by/12,
    levtype = pl,
    level = 1000,
    param = 129.128,
    target = "$SCRATCH/z1000_c.grb"

4) Fix errors in request:

retrieve,
    class = od,
    date = -7,
    expver = 1,
    levelist = 500/700/1000,
    levtype = pl,
    param = 130.128,
    step = 83/to/93/by/1,
    stream = oper,
    time = 12:00:00,
    type = fc,
    target = "$SCRATCH/temp.pl.grb"

This request returns some errors:

mars - ERROR  - 20231030.070854 - Last error is -43
mars - ERROR  - 20231030.070854 - Expected 33, got 27.
mars - ERROR  - 20231030.070854 - Request failed

Retrieved number of fields is lower than requested. Use MARS catalogue to see which of requested values are not available in MARS. You can use eCcodes to list content of resulting GRIB file and compare it to request:

grib_ls temp.pl.grb

Fix the MARS request by removing non-existing values from the request and re-run it again.

retrieve,
    class = od,
    date = -7,
    expver = 1,
    levelist = 500/700/1000,
    levtype = pl,
    param = 130.128,
    step = 83/84/85/86/87/88/89/90/93,
    stream = oper,
    time = 12:00:00,
    type = fc,
    target = "$SCRATCH/temp.pl.grb"

Post-processing

1)

Retrieve the 1000, 500 and 300 hPa temperatures valid for day before yesterday at noon from the 12 UTC forecast of four days ago on an N48 Gaussian Grid  > see in MARS Catalogue

...

retrieve,
    class = od,
    stream = oper,
    expver = 1, 
    type = forecast,
    param = 130.128,
    date = -4,
    time = 12,
    step = 48,
    grid = 1.5/1.5,
    levelist = 1000/500/300,
    levtype = pl,
    target = "$SCRATCH/t.ll.grb"

Retrieve Europe in European domain on the same way latitude-longitude grid as in the previous retrieval. Compare the size of the resulting file with the previous one. > see in MARS Catalogue

...

How many Latitude and Longitude point are found in European domain with 1.5 * 1.5 degrees resolution?

How many Latitude and Longitude point are found in European domain with 1.5 * 1.5 degrees resolution?

2)

Retrieve surface winds for previous week's 24 hour forecast on the European area converting to a 0.57 by 0.57 degrees latitude/longitude grid. > see in MARS Catalogue
What happens to the conversion?

2)

Retrieve surface winds for previous week's 24 hour forecast on the European area converting to a 0.57 by 0.57 degrees latitude/longitude grid. > see in MARS Catalogue
What happens to the conversion?

retrieve,
    class=od,
    date=-7,
    expver=1,
    levtype=sfc,
    param=165.128/166.128,
    step=24,
    stream=oper,
    time=12:00:00,
    type=fc,
    area = europe,                  # As the area is not compatible with the grid,
    grid = 0.57/0.57,               # MARS will expand the area to make it fit.
    target = "$SCRATCH/10u_10v.grb"

With grib_dump you will see that the area in 10u_10v.grb and previously downloaded file from European area: t.ll.europe.grb are not identical.

Advanced MARS usage

Multi-target

Ensemble forecasts

1)

Retrieve the 2 metre temperature and 2 metre dew point temperature fields of the first 10 members of a week old 12 UTC ensemble prediction (type: perturbed forecast) for 2, 3 and 4 days in advance > see in MARS Catalogue

retrieve,
    class=od,
    date=-7,
    expver=1,
    levtype=sfc,
    number=1/to/10,
    param=167.128/168.128,
    step=48/to/96/by/24,
    stream=enfo,
    time=12,
    type=pf,

retrieve,
    class=od,
    date=-7,
    expver=1,
    levtype=sfc,
    param=165.128/166.128,
    step=24,
    stream=oper,
    time=12:00:00,
    type=fc,
    area = europe,                  # As the area is not compatible with the grid,
    grid = 0.57/0.57,               # MARS will expand the area to make it fit.
    target = "$SCRATCH/10u_10v.grb"

With grib_dump you will see that the area in 10u_10v.grb and previously downloaded file from European area: t.ll.europe.grb are not identical.

Ensemble forecasts

1)

Retrieve the 2 metre temperature field of the first 10 members of a week old 12 UTC ensemble prediction (type: perturbed forecast) for 2, 3 and 4 days in advance > see in MARS Catalogue

ensemble.[type].{shortName}.grb"

2)

Retrieve the same for the control forecast. > see in MARS Catalogue

retrieve,
    class=od,
    date=-7,
    expver=1,
    levtype=sfc,
    param=167.128/168.128,
    step=48/to/96/by/24,
    stream=enfo

retrieve,
    classtime=od12,
    datetype=-7cf,
      expver=1,
    levtype=sfc,
    number=1/to/10,
    param=167.128,
    step=48/to/96/by/24,
    stream=enfo,
    time=12,
    type=pf,
    target = "$SCRATCH/ensemble.[type].{shortName}.grb"

2)

Retrieve the same for the control forecast. > see in MARS Catalogue

retrieve,
    class=od,
    date=-7,
    expver=1,
    levtype=sfc,
    param=167.128,
    step=48/to/96/by/24,
    stream=enfo,
    time=12,
    type=cf,
    target = "$SCRATCH/ensemble.[type].{shortName}.grb"

Note that this time we user MARS keyword "type" and ecCodes key "shortName" in the name of the target file. Add one more parameter "param" (find in MARS catalogue link which parameters are available) to the second request and re-run it again. How many target files did the request create?

Advanced MARS usage

Efficiency

In one of previous' requests output find a line looking like this:

mars - INFO   - ............... - The efficiency of your requests in the last 12 hours is

What does it say, what is efficiency of your MARS usage?

This score is based on number of times you access the same data tape during 12 hours. If you access the same tape more than once, the score will be lower than 100%. More accesses to the same tame - lower the score is. Conclusion: One should download all required data from a single tape file in one go to avoid queueing for the same tame multiple times!

List 

target = "$SCRATCH/ensemble.[type].{shortName}.grb"

Note that this time we used the MARS keyword "type" and the ecCodes key "shortName" in the name of the target file. How many target files did the requests create?

Efficiency

In one of previous' requests output find a line looking like this:

mars - INFO   - ............... - The efficiency of your requests in the last 12 hours is

What does it say, what is efficiency of your MARS usage?

This score is based on number of times you access the same data tape during 12 hours. If you access the same tape more than once, the score will be lower than 100%. More accesses to the same tame - lower the score is. Conclusion: One should download all required data from a single tape file in one go to avoid queueing for the same tame multiple times!

Question: You need to download selection of surface parameters from operational analysis at 12 UTC, every day for two consecutive months September and October 2021. How many requests are you going to run?

Tip: Use MARS catalogue to see which analysis are stored together in a tape file.

List 

Find the request size of for control forecast for a historical date without downloading the data:

list,
    class=od,
    date=2020-01-01,
    expver=1,
    levtype=sfc,
    param=167.128,
    step=1/to/96/by/1,
    stream=enfo,
    time=12,
    type=cf

Since this only query MARS database without downloading the data and running interpolation, the request can be run interactively on the login node:

mars myrequestfile

instead of using batch system.

What is the total size of requested data?

To find which portion of requested data is available online vs on tape, print the request "cost"Find the request size of for control forecast for a historical date without downloading the data:

list,
    output=cost,
    class=od,
    date=2020-01-01,
    expver=1,
    levtype=sfc,
    param=167.128,
    step=1/to/96/by/1,
    stream=enfo,
    time=12,
    type=cf

Since this only query MARS database without downloading the data and running interpolation, the request can be run interactively on the login node:

mars myrequestfile

instead of using batch system.

=12,
    type=cf

Is the data available on-line or on tapes onlyWhat is the total size of requested data?

Run the same request for 7 daysTo find which portion of requested data is available online vs on tape, print the request "cost":

list,
    output=cost,
    class=od,
    date=2020-01-01/to/2020-01-07,
    expver=1,
    levtype=sfc,
    param=167.128,
    step=1/to/96/by/1,
    stream=enfo,
    time=12,
    type=cf

Is the data available on-line or on tapes only?

 type=cf

On how many tapes is the data distributed now?

Run the same request for 7 daysHow about entire month of such data:

list,
    output=cost,
    class=od,
    date=2020-01-01/to/2020-01-07=2020-01-01/to/2020-01-31,
    expver=1,
    levtype=sfc,
    param=167.128,
    step=1/to/96/by/1,
    expverstream=1enfo,
    levtypetime=sfc12,
    param=167.128,
    step=1/to/96/by/1type=cf

If you need to download this data for entire January 2020, how would you proceed:

• Running one request for entire month (pros / cons)
• Running 31 daily requests (pros / cons)

?

Read, Write, Compute

Retrieve U and V wind components at 10 metre from 10 day old analysis and calculate wind speed > see in MARS catalogue 

retrieve,
    streamclass =enfo od,
    expver time=12,
    type=cf

On how many tapes is the data distributed now?

How about entire month of such data:

list 1,
    outputstream =cost oper, 
    type class=od analysis,
    date =2020-01-01/to/2020-01-31 -10,
    time expver=1 12,
    levtype = sfc,
    param =167 165.128, # u
    grid step=1/to/96/by/1 1/1,
    fieldset = u
retrieve,
    stream=enfo,
    time=12,
    type=cf

If you need to download this data for entire January 2020, how would you proceed:

• Running one request for entire month (pros / cons)
• Running 31 daily requests (pros / cons)

Read, Write, Compute

Retrieve U and V wind components at 10 metre from 10 day old analysis and calculate wind speed > see in MARS catalogue 

retrieveparam = 166.128, # v
	fieldset = v
compute,
    formula = "sqrt(u*u + v*v)",
    classfieldset = odspeed
write,	
    expverfieldset = 1speed,
    streamtarget = oper, "windspeed"

Additional exercise (question from Nikolaos):

This request will read previously created GRIB file, crop the European area from global domain and save it as another GRIB file:

read,
    typesource = analysis"windspeed",
    dategrid = -101/1,
    timearea = 12europe,
    levtypetarget = sfc,
    param = 165.128, # u
    grid = 1/1,
    fieldset = u
retrieve,
    param = 166.128, # v
	fieldset = v
compute,
    formula = "sqrt(u*u + v*v)",
    fieldset = speed
write,	
    fieldset = speed,
    target = "$SCRATCH/windspeed""europe_windspeed"

To convert previously created GRIB file to netCDF using ecCodes in a terminal on the ECS or HPC:

>module load ecmwf-toolbox
>grib_to_netcdf -o windspeed.nc windspeed

grib_to_netcdf: Version 2.30.2
grib_to_netcdf: Processing input file 'windspeed'.
grib_to_netcdf: Found 1 GRIB field in 2 files.
grib_to_netcdf: Ignoring key(s): method, type, stream, refdate, hdate
grib_to_netcdf: Creating netCDF file 'windspeed.nc'
grib_to_netcdf: NetCDF library version: 4.9.1 of Feb  9 2023 13:54:09 $
grib_to_netcdf: Creating large (64 bit) file format.
grib_to_netcdf: Defining variable 'u10'.
grib_to_netcdf: Done.