...
- 1 file per month of Analysis (1 type of level, all times, levels, and parameters), for example, have a look in the MARS Catalogue for for February 2015 model level analysis
- 1 file per Forecast (1 type of level, all steps, levels and parameters), for example, have a look in the MARS Catalogue for 1st February 2015 model level forecast at 12 UTC
- 1 file per Ensemble Forecast (1 type of level, all steps, members, levels and parameters), for example, have a look in the MARS Catalogue for 1st February 2015 pressure level 50 member ensemble at 12 UTC
...
| Gliffy Diagram | ||||
|---|---|---|---|---|
|
Post-processing
Any requested data manipulation or post-processing is carried out by the MARS client, except in the case of a local Member State's the Web API client where data is first processed at ECMWF prior to its transmission over the network. The post-processing is carried out by MIR, a set of routines present in the EMOSLIB library. Please, refer to the Field interpolation software routines for in-depth details about such post-processing.library of routines for Meteorological Interpolation and Regridding (see ECMWF newsletter no.152 for a description). For details on the various operations please see also the relevant post-processing keyword descriptions.
Sub-area extraction
Most of the data at ECMWF is global. A sub-area can be created using the area keyword by defining its latitude and longitude boundaries: North/West/South/East.
...
GRIB coded fields have a specified number of bits per packed value which can be changed with keyword accuracy. This might be useful when trying to retrieve fields from MARS identical to those you get from dissemination.
...
By default spectral fields are automatically truncated before interpolation to grid fields to reduce data volumes and spurious aliased values. When automatic truncation is applied MARS issues a warning message like INTFB: Resolution automatically set to .... The truncation can be controlled using the resol keyword. Default truncations are detailed in the following table.
| Default spectral truncation wave number for given lat/lon grid increment ∆. | |
| Grid increment [deg] | Truncation |
| 2.5 ≤ ∆ | T63 |
| 1.5 ≤ ∆ < 2.5 | T106 |
| 0.6 ≤ ∆ < 1.5 | T213 |
| 0.4 ≤ ∆ < 0.6 | T319 |
| 0.3 ≤ ∆ < 0.4 | T511 |
| 0.15 ≤ ∆ < 0.3 | T799 |
| 0.09 ≤ ∆ < 0.15 | T1279 |
| 0.0 ≤ ∆ < 0.09 | T2047 |
keywords truncation and intgrid. Users wanting to post-process at the full archived resolution can specify resol truncation = avnone in the request.
Note that high resolutions might need more resources to carry out post-processing.
...
Fields on spherical harmonics or Gaussian grid can be rotated with the keyword rotation. The rotation is performed prior to any other conversion. Therefore:
- Spherical harmonics fields can be retrieved as rotated spherical harmonics or converted to rotated latitude/longitude or Gaussian grid fields.
- Gaussian grid fields can be retrieved as rotated latitude/longitude or rotated Gaussian grid fields.
If the input is a spectral field and the output is a rotated spectral field, a file of rotation coefficients is generated and used in the processing. The convergence of the algorithm for rotating the spectral fields is sensitive to the spectral truncation. For some high resolutions and some large angles of rotation, it is necessary to split the rotation into three steps (two forwards and one backwards!). The size of the rotation coefficient files can be very large.
If the input is a spectral field and the output is a rotated grid point field, the spectral field is interpolated to a non-rotated Gaussian grid which is then transformed to the required rotated grid point field using a 12-point interpolation based on the FULL-POS scheme used in the ECMWF forecast model.
A rotated grid-point field is created from an input grid-point field by finding for each rotated grid-point its nearest four neighbours in the input field and carrying out a bilinear interpolation.
Retrieval efficiency
Retrieval efficiency
To retrieve data efficiently please follow the hints belowIn order to retrieve data efficiently users should follow the following hints:
- Whenever possible, use a local file system to store the target file (e.g. $SCRATCH on ecgate for external users) as this will avoid unnecessary network traffic.
- Estimate the data volume to be retrieved before issuing a request. It is easy to retrieve Gigabytes of data. Check that computer resources and limits are adequate for the amount of data to retrieve/interpolate in order to avoid unnecessary processing (MARS will fail if a quota is exceeded or in the case of any Unix problem regarding resources such as memory or CPU time).
- Estimate the number of fields to be retrieved before issuing a request. Try to retrieve a sensible number, up to tens of thousands of fields.
- Reduce the number of tapes involved. The number of tapes a MARS request is going to access will have an impact on its scheduling on the server. As a rule of thumb, two or more separate requests accessing files on different tapes are scheduled more efficiently than a single request accessing two or more tapes. A large number of tapes implies more waiting time. Use multiple requests in one MARS call whereby all the data is written to one output file.
- When retrieving large datasets (e.g. Re-Analysis), try to retrieve as many data from the same tape file as possible. Then, avoid caching the data on the server by specifying use = infrequent.
- Avoid constantly accessing the same tape. If you issue a large number of requests, all accessing data on the same tape, this can keep that tape in the drive for many hours and potentially cause some damage. Once you access a tape read as many fields as possible and if needed split the output by level, parameter etc. using the multi-target feature. This would reduce the amount of requests and make your extractions much faster.
...
Please also follow the Guidelines to write efficient MARS requests.
Troubleshooting
Syntactic errors
If a comma is missing or an unknown MARS keyword is specified, MARS will stop processing your request and report a syntax error.
...
Messages starting with Failed HPSS call: n = ::hpss_Read(fd_,buffer,... are HPSS errors. The actual message text can vary but they usually mean that the data is unavailable from tapes for some time. These errors are passed to the client and make your request fail. As a rule, after checking that there is no ongoing system session, re-run your request before reporting the error. If the failure is consistent, please inform ECMWF's Call Service Desk.
Assertion failed
Error messages starting with Assertion failed: followed by offset[i] > offset[i-1] or handle[n] != 0 hint at an attempt to retrieve the same field more than once, e.g. by specifying param=z/t/z or step=24/48/24. The actual message varies depending on how exactly the data is retrieved. The general advice in such cases is to check the request for multiple occurrences of keyword values.
...