The MARS system is based on a client/server architecture, which makes it more flexible and adaptive to changes in a complex environment like ECMWF's.
The interaction between clients and servers is based on MARS requests. Clients send MARS requests to a server which looks for the data online or off-line, depending on the capabilities of the given server. If it fails to satisfy the request, the client will contact another server, according to a list of pre-configured servers.
Clients
The MARS client is a C program linked with GRIB API and the EMOSLIB library (which contains the GRIB interpolation software). The MARS source code is also embedded in other applications, such as Metview.
Access at ECMWF
MARS can be executed either in batch or interactive mode. Usually, clients issue requests from:
- Workstations and servers, suitable for most retrievals
- Supercomputers, retrieving data to be used as input for models
- Metview, for interactive/batch visualisation/plotting and manipulation
It is recommended to use workstations or workstation servers for data retrieval instead of the supercomputers. This avoids unnecessary idle time when data has to be read from tape.
Member States/Co-operating States access
Most of the access from Member State users comes via the ecgate system, by logging in or submitting batch jobs to ECMWF's computers.
Moreover, the MARS client software is also available to access ECMWF's MARS archive system remotely by installing it at Member State sites. It removes user's need to login. Access and transfer of the data is transparent to the user, although data transfer may be slow. In addition, it is possible to create secondary cache systems at Member State sites in order to minimise remote access and transfer for popular data.
Servers
Reports Data Base
The RDB Contains online observations received via ECMWF's acquisition system. This system has been interfaced with MARS to allow real-time observation access. Access to this server is meant for monitoring and operational archive purpose only.
Fields Data Base (FDB)
This is where models running at ECMWF write their outputs. It contains data produced by the most recent cycles.
Depending on the configuration and disk resources, it can contain up to several days of operational data and more recent research experiments. Also, there are many Fields Databases, several per supercomputer or server able to run ECMWF's models.
It is meant to provide very fast access as all the data resides online. This makes it very suitable for model input data retrieval or last cycles data access.
Main Archive
This is the core of the MARS System and consists of the following hardware and software:
Hardware
- Dedicated multiprocessor servers
- Several Terabytes of disk space, used for temporary storage before data is written to tape and for caching purposes while data is read from tape
- A set of automated tape libraries
Software
- A set of applications written in C++ linked with the EMOSLIB library
- High Performance Storage System (HPSS), which controls the tape, tape drive and robotic related activities.
Some characteristics of the Main Archive system such as request scheduling and data collocation are very important for users in order to optimise data retrieval.
Interaction client/server: request execution
The MARS client has a configuration of servers to access when looking for data. The following is a schematic view of actions the MARS client performs per request:
- Checks the request syntax with the help of the MARS language
- Prints the request to be processed and the number of fields expected
- Queries the cache system (if configured)
- Queries all supercomputers' Fields Data Bases (if the data are not cached)
- Queries the main archives (if data are not in the FDB)
- Transfers the data to the client and post-processes (if needed)
- Caches the data (if the cache is present)
- Reports on the result
Note that post-processing is done while the data is being transferred and before writing to disk.