Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Code Block
languagepy
request = mars.Request(
        ("HHL", "DBZ"),
        date=ref_time.strftime("%Y%m%d"),
        time=ref_time.strftime("%H00"),
        expver="0001",
        levelist=tuple(range(1, 82)),
        number=tuple(range(11)),
        step=lead_time,
        levtype=mars.LevType.MODEL_LEVEL,
        model=mars.Model.COSMO_1E,
        stream=mars.Stream.ENS_FORECAST,
        type=mars.Type.ENS_MEMBER,
    )
    ds = model_data.get(request, ref_param_for_grid="HHL")


An important component of the FDB framework is the FDB remote. This new development of the FDB family allows to serve FDB data to requests that do not have access to the Lustre based deployment of FDB. 

The FDB remote is implemented as a set of services, including a catalogue (for querying data) and a store (that serves data requests).

A typical use case that needs the deployment of the FDB remote is the access of ECMWF production data from the EWC environment.

One of the architecture designs of pilot project is based on access to ECMWF production directly from EWC (without the need to setup and maintain dissemination streams). However, EWC is not in the same network as the HPC production of ECMWF. Applications running on EWC can not acccess the main FDB of the HPC center for retrieving data. FDB remote will be setup, such that any application in EWC will be able to request data to the server.

A simplified design of this deployment is shown in the following diagram: 




FDB performance


Jupyter Viewer
notebookUrlhttps://github.com/MeteoSwiss-APN/fdb-tools/blob/benchmarking/FDB/benchmarking/fdb-bench-results.ipynb