Open a terminal and login to the ECS login node following practicals covered on Monday (Teleport or VDI) cd to HPCPERM folder:
Code Block | ||
---|---|---|
| ||
ssh ecs-login cd $SCRATCH |
On ECS/HPC "mars" command is installed as part of MARS client and can be called without loading any module. The solutions below provide valid MARS requests which can be passed to the mars command as:
Code Block | ||
---|---|---|
| ||
mars < myrequestfile # or mars myrequestfile |
Table of Contents
Basic MARS usage
Simple retrieves
1)
Retrieve 1000,850,700 and 500 hPa temperatures from the 15th of October 2023 12 UTC Analysis > see in MARS Catalogue
Expand | ||
---|---|---|
| ||
retrieve, class = od, |
Check the request output. Any "ERROR" or "WARN" messages? If not, check if the target file exists:
Code Block | ||
---|---|---|
| ||
ls t.grb |
This morning you learned how to use ecCodes. You can use it now to check content of your GRIB file:
Code Block | ||
---|---|---|
| ||
module load ecmwf-toolbox grib_ls t.grb |
How many messages does the GRIB file contain? Does it match the number of expected fields retrieved from MARS?
Expand | ||
---|---|---|
| ||
grib_ls t.grb edition centre typeOfLevel level dataDate stepRange dataType shortName packingType gridType 4 of 4 messages in t.grb |
2)
We run the first MARS request on a login node of ECS. Because of the limitations imposed on interactive sessions (e.g., memory, cpu time) it is better to use batch jobs to run MARS request which can be large with long runtime. Yesterday, you learned how to run fractional jobs. Let's use it know to execute next MARS request. The following template script can be used for this purpose:
Code Block | ||||
---|---|---|---|---|
| ||||
#!/bin/bash #SBATCH --job-name=mars_job #SBATCH --qos=ef #SBATCH --time=1:00:00 #SBATCH --output=mars.%j.out #SBATCH --error=mars.%j.out # For larger requests downloading data from tapes # extend walltime "--time" to 1 day or longer set -xe # Enable multi target option: export MARS_MULTITARGET_STRICT_FORMAT=1 export SCRATCH export PERM mars <<EOF #{paste your MARS request here} EOF |
More examples can be found on: HPC2020: example Slurm serial batch job scripts for ECS
Use the template above to retrieve wind components valid for 5 days ago at 12:00 and 18:00 from forecast made 7 days ago at 12 UTC at all 137 model levels > see in MARS Catalogue
Expand | ||
---|---|---|
| ||
retrieve, |
Submit your MARS job with:
Code Block | ||
---|---|---|
| ||
ecsbatch mars_job.sh |
You can check the job status with:
Code Block | ||
---|---|---|
| ||
watch ecsqueue -u $USER |
once the MARS job is complete, please check the job's output file, read MARS messages, check for any ERROR or WARNING messages. Read "INFO" messages, which action has taken longest time:
Expand | ||
---|---|---|
| ||
mars - INFO - 20231029.184331 - Transfering 1798291592 bytes mars - INFO - 20231029.184333 - Deriving U and V from vorticity and divergence mars - INFO - 20231029.184404 - 548 fields retrieved from 'marsod' |
From now on, we will use mars_job.sh template to run all MARS requests as batch jobs.
3)
Use the same batch job template (mars_job.sh) from example 2) to run MARS request retrieving the 1000 hPa geopotential from two weeks old 12 UTC forecast (> see in MARS Catalogue) steps: 12 hour to 240 hour with 12 hour intervals:
Expand | ||
---|---|---|
| ||
retrieve, |
4) Fix errors in request:
Code Block | ||
---|---|---|
| ||
retrieve, class = od, date = -7, expver = 1, levelist = 500/700/1000, levtype = pl, param = 130.128, step = 83/to/93/by/1, stream = oper, time = 12:00:00, type = fc, target = "$SCRATCH/temp.pl.grb" |
This request returns some errors:
Code Block | ||
---|---|---|
| ||
mars - ERROR - 20231030.070854 - Last error is -43 mars - ERROR - 20231030.070854 - Expected 33, got 27. mars - ERROR - 20231030.070854 - Request failed |
Retrieved number of fields is lower than requested. Use MARS catalogue to see which of requested values are not available in MARS. You can use eCcodes to list content of resulting GRIB file and compare it to request:
Code Block | ||
---|---|---|
| ||
grib_ls temp.pl.grb |
Fix the MARS request by removing non-existing values from the request and re-run it again.
Expand | |||||
---|---|---|---|---|---|
| |||||
After step 90, steps are archived on 3 hours. Steps 91 and 92 do not exist in MARS.
|
Post-processing
1)
Retrieve the 1000, 500 and 300 hPa temperatures valid for day before yesterday at noon from the 12 UTC forecast of four days ago on an N48 Gaussian Grid > see in MARS Catalogue
Expand | ||
---|---|---|
| ||
retrieve, |
Retrieve the same on a latitude/longitude grid, 1.5 by 1.5 degrees. > see in MARS Catalogue
Expand | ||
---|---|---|
| ||
retrieve, |
Retrieve European domain on the same latitude-longitude grid as in the previous retrieval. Compare the size of the resulting file with the previous one. > see in MARS Catalogue
Expand | ||
---|---|---|
| ||
retrieve, |
Use grib_ls to print grid type for each of the three GRIB files download in above requests. Is the resulting grid expected?
Expand | ||
---|---|---|
| ||
module load ecmwf-toolbox grib_ls t.gg.grb t.gg.grb edition centre typeOfLevel level dataDate stepRange dataType shortName packingType gridType 1 ecmf isobaricInhPa 1000 20231025 48 fc t grid_simple reduced_gg grib_ls t.ll.grb t.ll.grb edition centre typeOfLevel level dataDate stepRange dataType shortName packingType gridType 1 ecmf isobaricInhPa 1000 20231025 48 fc t grid_simple regular_ll |
How many Latitude and Longitude point are found in European domain with 1.5 * 1.5 degrees resolution?
Expand | ||
---|---|---|
| ||
grib_dump t.europe.grb | less ... Ni = 49; Nj = 28; ... |
2)
Retrieve surface winds for previous week's 24 hour forecast on the European area converting to a 0.57 by 0.57 degrees latitude/longitude grid. > see in MARS Catalogue
What happens to the conversion?
Expand | ||
---|---|---|
| ||
retrieve, class=od, date=-7, expver=1, levtype=sfc, param=165.128/166.128, step=24, stream=oper, time=12:00:00, type=fc, |
With grib_dump you will see that the area in 10u_10v.grb and previously downloaded file from European area: t.ll.europe.grb are not identical.
Advanced MARS usage
Multi-target
Ensemble forecasts
1)
Retrieve the 2 metre temperature and 2 metre dew point temperature fields of the first 10 members of a week old 12 UTC ensemble prediction (type: perturbed forecast) for 2, 3 and 4 days in advance > see in MARS Catalogue
Expand | ||
---|---|---|
| ||
retrieve, |
2)
Retrieve the same for the control forecast. > see in MARS Catalogue
Expand | ||
---|---|---|
| ||
retrieve, |
Note that this time we user MARS keyword "type" and ecCodes key "shortName" in the name of the target file. How many target files did the requests create?
Expand | ||
---|---|---|
| ||
2 target files |
Efficiency
In one of previous' requests output find a line looking like this:
Code Block | ||
---|---|---|
| ||
mars - INFO - ............... - The efficiency of your requests in the last 12 hours is |
What does it say, what is efficiency of your MARS usage?
This score is based on number of times you access the same data tape during 12 hours. If you access the same tape more than once, the score will be lower than 100%. More accesses to the same tame - lower the score is. Conclusion: One should download all required data from a single tape file in one go to avoid queueing for the same tame multiple times!
Question: You need to download selection of surface parameters from operational analysis at 12 UTC, every day for two consecutive months September and October 2021. How many requests are you going to run?
Tip: Use MARS catalogue to see which analysis are stored together in a tape file.
Expand | ||
---|---|---|
| ||
An entire month of operational analysis for selected level type is stored in one tape file. Consequently, all needed analysis for one month should be download in one request to avoid queuing for the same tape multiple times. Two months are download in two requests, one for each month. |
List
Find the request size of for control forecast for a historical date without downloading the data:
Code Block | ||
---|---|---|
| ||
list, class=od, date=2020-01-01, expver=1, levtype=sfc, param=167.128, step=1/to/96/by/1, stream=enfo, time=12, type=cf |
Since this only query MARS database without downloading the data and running interpolation, the request can be run interactively on the login node:
Code Block | ||
---|---|---|
| ||
mars myrequestfile |
instead of using batch system.
What is the total size of requested data?
Expand | ||
---|---|---|
| ||
Grand Total: ============ Entries : 92 Total : 305,955,384 (291.782 Mbytes) mars - INFO - 20231029.195448 - Request performed on database 'marsod' mars - INFO - 20231029.195448 - Memory used: 31.71 Mbyte(s) mars - INFO - 20231029.195448 - No errors reported |
To find which portion of requested data is available online vs on tape, print the request "cost":
Code Block | ||
---|---|---|
| ||
list, output=cost, class=od, date=2020-01-01, expver=1, levtype=sfc, param=167.128, step=1/to/96/by/1, stream=enfo, time=12, type=cf |
Is the data available on-line or on tapes only?
Expand | ||
---|---|---|
| ||
.... size=305955384; number_of_fields=92; online_size=0; off_line_size=305955384; number_of_tape_files=1; number_of_disk_files=0; number_of_online_fields=0; number_of_offline_fields=92; number_of_tapes=1; ... |
Run the same request for 7 days:
Code Block | ||
---|---|---|
| ||
list, output=cost, class=od, date=2020-01-01/to/2020-01-07, expver=1, levtype=sfc, param=167.128, step=1/to/96/by/1, stream=enfo, time=12, type=cf |
On how many tapes is the data distributed now?
Expand | ||
---|---|---|
| ||
... number_of_tapes=5; ... |
How about entire month of such data:
Code Block | ||
---|---|---|
| ||
list, output=cost, class=od, date=2020-01-01/to/2020-01-31, expver=1, levtype=sfc, param=167.128, step=1/to/96/by/1, stream=enfo, time=12, type=cf |
Expand | ||
---|---|---|
| ||
number_of_tapes=11; |
If you need to download this data for entire January 2020, how would you proceed:
- Running one request for entire month (pros / cons)
- Running 31 daily requests (pros / cons)
?
Read, Write, Compute
Retrieve U and V wind components at 10 metre from 10 day old analysis and calculate wind speed > see in MARS catalogue
Code Block | ||
---|---|---|
| ||
retrieve, class = od, expver = 1, stream = oper, type = analysis, date = -10, time = 12, levtype = sfc, param = 165.128, # u grid = 1/1, fieldset = u retrieve, param = 166.128, # v fieldset = v compute, formula = "sqrt(u*u + v*v)", fieldset = speed write, fieldset = speed, target = "windspeed" |
Additional exercise (question from Nikolaos):
This request will read previously created GRIB file, crop the European area from global domain and save it as another GRIB file:
Code Block | ||
---|---|---|
| ||
read, source = "windspeed", grid = 1/1, area = europe, target = "europe_windspeed" |
To convert previously created GRIB file to netCDF using ecCodes in a terminal on the ECS or HPC:
Code Block | ||
---|---|---|
| ||
>module load ecmwf-toolbox >grib_to_netcdf -o windspeed.nc windspeed grib_to_netcdf: Version 2.30.2 grib_to_netcdf: Processing input file 'windspeed'. grib_to_netcdf: Found 1 GRIB field in 2 files. grib_to_netcdf: Ignoring key(s): method, type, stream, refdate, hdate grib_to_netcdf: Creating netCDF file 'windspeed.nc' grib_to_netcdf: NetCDF library version: 4.9.1 of Feb 9 2023 13:54:09 $ grib_to_netcdf: Creating large (64 bit) file format. grib_to_netcdf: Defining variable 'u10'. grib_to_netcdf: Done. |