...
Moreover, the MARS client software is also available to access ECMWF's MARS archive system remotely by installing it at Member State sites. It removes user's need to login. Access and transfer of the data is transparent to the user, although data transfer may be slow. In addition, it is possible to create secondary cache systems at Member State sites in order to minimise remote access and transfer for popular data.
Servers
Reports Data Base
The RDB Contains online observations received via ECMWF's acquisition system. This system has been interfaced with MARS to allow real-time observation access. Access to this server is meant for monitoring and operational archive purpose only.
Fields Data Base (FDB)
This is where models running at ECMWF write their outputs. It contains data produced by the most recent cycles.
Depending on the configuration and disk resources, it can contain up to several days of operational data and more recent research experiments. Also, there are many Fields Databases, several per supercomputer or server able to run ECMWF's models.
Details about the architecture of the MARS server are give in a separate article. MARS has evolved since this article was written back in 1999, e.g., software developed at ECMWF has replaced ObjectStore as metadata manager, but the general architecture is still valid. In 2003, the MARS contents were migrated to the High Performance Storage System (HPSS) tape management software, which replaced Tivoli Storage Manager (TSM, previously known as ADSM).
The various servers are described belowIt is meant to provide very fast access as all the data resides online. This makes it very suitable for model input data retrieval or last cycles data access.
Main Archive
This is the core of the MARS System and consists of the following hardware and software:
...
Some characteristics of the Main Archive system such as request scheduling and data collocation are very important for users in order to optimise data retrieval.
Fields Data Base (FDB)
This is where models running at ECMWF write their outputs. It contains data produced by the most recent cycles.
Depending on the configuration and disk resources, it can contain up to several days of operational data and more recent research experiments. Also, there are many Fields Databases, several per supercomputer or server able to run ECMWF's models.
It is meant to provide very fast access as all the data resides online. This makes it very suitable for model input data retrieval or last cycles data access.
Reports Data Base
The RDB Contains online observations received via ECMWF's acquisition system. This system has been interfaced with MARS to allow real-time observation access. Access to this server is meant for monitoring and operational archive purpose only.
Interaction client/server: request execution
...
Note that post-processing is done while the data is being transferred and before writing to disk.
MARS migration to GRIB API
The MARS client using GRIB API as its data decoder instead of GRIBEX has been made the default on all major platforms on 10 May 2011. To retrieve data in GRIB 2 format the use of this version is essential.
| Expand | |||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| |||||||||||||||
The MARS application has been migrated to use GRIB API as its data decoder instead of GRIBEX. This migration has two components:
Because the interpolation software (libemos) uses extensively GRIBEX, many parts of the code had to be modified. Please see Field interpolation software for more information. For the migration of the MARS client we have tried, as much as possible, to avoid changes to the GRIB headers or the results. Extensive tests have been run and comparisons between interpolations of MARS/gribex and MARS/grib_api have been performed using
where the options have been chosen to skip differences smaller than twice the packing error. However, in some cases we have been unable to produce the same results. At the same time, we have found features of libemos/gribex (libemos using GRIBEX as data decoder) that have been corrected in libemos/grib_api (libemos using GRIB API as data decoder). Please see below for more details on known differences between MARS/gribex and MARS/grib_api. MARS client for installation in Member StatesFor a grib_api MARS client version 20110322 or higher should be used, which needs to be compiled with libemos 381 (or higher) and grib_api 1.9.9 (or higher), which can be freely downloaded. Known differences between MARS/gribex and MARS/grib_apiInterpolation of wave mediterranean fieldsThe interpolation of wave fields has been re-written. The new routine correctly recognizes sea points on the southernmost, last latitude for limited area wave fields. This has been verified by the wave experts. Libemos/gribex has not been fixed. Bitmaps and framesMARS/gribex doesn't initialise bitmaps (which are also used for frames), while MARS/grib_api initialises bitmaps to 0. Since bitmap arrays are filled differently, for bitmaps that do not fit a multiple of 8, the remaining bits can have different values. One can detect this difference by running grib_compare, which produces the following output: Different behaviour in MARS (not necessarily interpolation)
|