The OpenIFS Data Hub is a facility to generate initial and boundary conditions for OpenIFS forecast model experiments. It is available only to users of the IFS or the OpenIFS model.

The Data Hub does not produce any OpenIFS model diagnostics or forecast data. It generates data that is required to carry out OpenIFS model experiments.

If you are interested in ECMWF data products or need more general information about ECMWF data please follow this link or contact ECMWF Support. 

Latest Updates

What is the OpenIFS Data Hub?

One of the most pressing needs OpenIFS users face is to create initial experiment data (initial and boundary conditions) to start a model experiment. Before the Data Hub existed such initial experiment data were requested by contacting OpenIFS support via email. 

The OpenIFS Data Hub allows OpenIFS and IFS users to produce their own initial experiment data through a web-based application that triggers an automatic data generation process on our servers. The initial data files are produced using the exact same method as if the OpenIFS team had created the data. At the end of the process the user is provided with a download web link to retrieve the requested data files.

The Data Hub therefore removes the need to contact the OpenIFS team at ECMWF and provides one common system that is available to both ECMWF staff and external OpenIFS users everywhere. Any initial data types that are currently not yet served by the Data Hub can be requested, as before, from OpenIFS support.  

What data products does the Data Hub provide?

The OpenIFS Data Hub provides data products that are required to carry out experiments with OpenIFS models. These include:

1. Initial and boundary conditions for global 3D meteorological forecast experiments.
2. As above, however additionally including initial and boundary data to simulate interactively atmospheric composition.
3. Relaxation files to run OpenIFS in nudged configuration.
4. Initial and forcing data to run experiments with the OpenIFS Single-Column Model.

Please note that the Data Hub only provides input data for the model. It does not carry out OpenIFS model experiments and hence does not produce OpenIFS model outputs.

Data licence information

The Data Hub generates data products that are designed for experiments with OpenIFS models.

Using the Data Hub and submitting data requests requires the user to consent to the terms of the OpenIFS Data Hub Licence Agreement. This agreement also provides details of the data licences for the various data sources which are used by the Data Hub to generate its data products. 

How to use the Data Hub

This section provides step by step information how to use the Hub to generate data products for use with OpenIFS.

Access and login

Please note that if you are registered with ECMWF as an OpenIFS user then you should already have access to the Data Hub.

The OpenIFS Data Hub can be accessed here:     

You will need an ECMWF user account to access the Data Hub.

The Dashboard page

After the initial user authentication you will reach the Data Hub's dashboard page which shows your data request history. 

To submit a new request for initial data press the "New Request" button at the top of the page. This will open a drop-down menu with several data product options:

1. Initial data - meteo only forecast: This will generate initial experiment data and time-varying boundary conditions for global 3D forecast experiments.
2. Initial data - atmos composition forecast: This will generate initial data, as above, with additional data for simulating atmospheric composition.
3. Relaxation files - nudged experiments: This will generate relaxation files which are required for model runs in nudged configuration. 
4. Initial data - single column model: This will generate initial experiment data and forcing data for the single-column model.

Clicking on either of these options will open a new web page containing a data request form.  

How to complete the data request form

The data request form is specific to the type of data product. If you need help to complete the form, please follow the links below where you can find a description of the input parameters for each form:

Submitting your data request

Once the information in the data request form has been completed, the request can be submitted to the Hub by pressing the Submit button.

Once this button has been pressed, the request information is added to the queue and the data generation process can begin. The request history dashboard page is shown. 

Some preliminary checking  is done while entering the request data.

Further consistency checks will be carried out once the request has been accepted. Note that any user choices resulting in an unworkable data requests will be identified early in the process and the user request will be aborted. 

A maximum data volume of 500,000 MB is permitted for a single request. Any requests exceeding this data volume will be aborted. For reference, an initial data request at the current operational grid resolution (Tco1279L137) for a single start date with 10 days forecast length has an (uncompressed) data volume of approximately 20,000 MB. 

Examples for completing the request form

First example:  You require initial data for a single start date, created from ERA5 reanalysis data.

• The field "Source for initial data" needs to display ERA5 reanalysis (which is the default value). 
• Ensure that both date fields "First start date" and "Last start date" are set to the same date. When both dates are the same, any value selected for "Time increment between forecasts" will be ignored.
• Note that ERA5 data may not yet be available for the most recent months. If in doubt, check the MARS Catalogue prior to submitting your request.
• Select the experiment start time (default is 00:00 UTC) and the forecast length. The forecast length value determines the number of days for boundary conditions data (e.g. SSTs) in the ICMCL*INIT file.
• Select the model cycle and the model grid parameters for your request.
• Press "Submit".

Second example:  You require initial data with multiple start dates, every 12 hours, created from historical operational archive data.

• The field "Source for initial data" should now be set to Operational archive. 
• Note that the default value for the start date is set to the most recent possible date for operational archive data (for a 24 hour forecast, i.e. forecast length set to 1 day).  
• Ensure that the "First start date" and "Last start date" are set to the appropriate range of dates for your request; they must not be the same value for multiple start dates.
• In this case the value for "Time increment between forecasts" will be evaluated. For instance, if you need 12-hourly gaps between forecast start dates you should set this to 12. 
• Complete all the other fields as in the first example.
• Press "Submit". 

Third example:  You require initial data for a forecast experiment with atmospheric composition.

• CAMS operational data for cycle 48r1 is only available since 28 June 2023, so no earlier start date can be selected.
• To obtain initial data for the tropospheric chemistry scheme TM5 (including initial data for the AER bulk aerosol scheme) select the appropriate option in the drop-down menu for the chemistry. If additionally initial data for the BASCOE stratospheric chemistry is required change the option accordingly.
• Note that currently no time-varying surface or aircraft emissions are provided. 

Recent start dates:  When experiment data is required for dates close to the present day, it may be necessary to check data availability prior to submitting the request.

• The Data Hub will not provide any data for future dates.
• When requesting data created from the operational historical archive, note that for recent start dates the forecast length needs to be adjusted accordingly to end at least 6 days prior to the current date. The same applies to CAMS operational analysis data. 
• When requesting data created from the ERA5 reanalysis, you need to check the MARS Catalogue for the available dates. The time gap between the available reanalysis data and the present day can vary between a few weeks and several months. 

Checking the request progress 

After submitting a request the dashboard page is displayed again showing the user's request history.

• A new request entry has appeared at the top of the request history.
  A limit for open data requests (having status "new" or "in progress") applies per user and once this limit is reached no further requests can be submitted until some of the user's requests have completed or aborted. 
• At regular time intervals the system checks for new data requests. Once the request has been picked up its status indicator updates to "processing".
• Each request line entry has on the right side of the page a link to a corresponding Details page.
  
  Clicking on this link will open another web page with information about the data request details and its generation progress.

On this Details Page for a selected request the following information is shown: 

• The Details box in the upper left area shows the main request properties and its current status. After completion a download link for the data tarball will be displayed. For relaxation files the link redirects to the ECPDS portal where the generated data can be inspected. 
  
  Note that the download link has an expiry time stamp. After this expiry time the download link will be no longer active, by this time the generated data is removed from the storage area and is not available for download any longer. 
• The Config box on the right side of the Details Page displays the parameters that were selected previously in the input form. 
• The Changelog box displays the status history of the request with the current status on top.  Any potential errors or other reasons for aborting the request will be shown here. 

How do I know when my data will be available?

• You will be notified by email at the address linked to your ECMWF user-ID once your data request has completed (or if the request has aborted).
• You can also login to the Data Hub dashboard page and manually check the progress of your requests. This is described also in the next section below. 
• Once completed, the data of your request is available for download for a limited time (see the expiry time stamp). After this expiry time the download link will be disabled and the produced data files can no longer be retrieved.
• The request history remains in your dashboard page for your reference, even after the data file download link has expired.  

1 - Downloading initial experiment data for OpenIFS and the Single-Column Model

Initial experiment data (both, for meteo-only and for meteo+atmos composition) is packaged into a single file tgz archive on the Data Hub staging area, ready for download. 

The Data Hub provides the download link that is displayed both on the Details Page for the request, and also on the Dashboard Page.

The download link will remain active until the request's expiry date which is shown on the Details Page. After the expiry date it is no longer possible to access the data file which will then be removed from the staging area. 

2 - Downloading relaxation files from ECPDS

Relaxation files are staged on the ECMWF Production Data Store (ECPDS) and all relaxation files (generated by any user) can be inspected there. The default lifetime of these files is 90 days from their time of upload.

Access to the ECPDS portal requires authentication. You should login with your ECMWF username. Instead of your password you will need to use a generated 6-digit TOTP one-time access security token. The token can be generated using a smartphone with an authenticator app.

To access the data click on the link in the request details page, or alternatively go to https://aux.ecmwf.int/ and authenticate with your ECMWF username and the TOTP token (as password).

The web link that is provided after the data request has completed will redirect to the ECPDS portal and (following user authentication) the generated data is stored there as individual files which can be downloaded through the web browser.

To download multiple files more efficiently you can also connect with and ftp client (e.g. ftp, sftp, lftp etc) to:  sftp aux.ecmwf.int 

After connecting to ECPDS with the ftp client, the relaxation files are located in:  odh/02/relax/<cycle>/ERA5_<grid-resolution>/<year>/

An example retrieval using sftp is shown below. In this example relaxation files were produced for the year 2024 at TL159 with 60 vertical model levels:

bash$ sftp aux.ecmwf.int
Authorized access only.

*********************************************
SFTP Server

Use this SFTP server to download/upload files
from the ECMWF Dissemination System (ECPDS).
*********************************************

user_id@136.156.238.66's password: (type your TOTP token here, press Enter)
Connected to aux.ecmwf.int.
sftp> cd odh/02/relax/48r1/ERA5_TL159L60/
sftp> ls
2024    (this is the directory that contains the relaxation files for 2024)
sftp> mget -R 2024
Fetching /odh/02/relax/48r1/ERA5_TL159L60/2024/ to 2024
Retrieving /odh/02/relax/48r1/ERA5_TL159L60/2024
/odh/02/relax/48r1/ERA5_TL159L60/2024/rlxmlsh2024010100 100% 6145KB  20.4MB/s   00:00
/odh/02/relax/48r1/ERA5_TL159L60/2024/rlxmlsh2024010106 100% 6145KB  18.7MB/s   00:00
/odh/02/relax/48r1/ERA5_TL159L60/2024/rlxmlsh2024010112 100% 6145KB  19.8MB/s   00:00
(... retrieve directory 2024 recursively ...)
sftp> bye

How can I get help?

• To get help and ask for information about the OpenIFS Data Hub please contact OpenIFS support. The OpenIFS Team will respond to your question as soon as possible. 
• Please note that questions about OpenIFS and the OpenIFS Data Hub can also be posted in the OpenIFS Data Hub section of the ECMWF Forum. This allows the entire user community to benefit, and to respond or provide help and advice for your question.
• We welcome feedback about the Data Hub and suggestions for additional features and improvements. Please also contact the OpenIFS Team via the email address above. Feedback is important to us to understand the needs of the user community.
• Please be aware that the OpenIFS Team cannot provide help and assistance for topics that are not related to OpenIFS. Queries about other ECMWF products (e.g. other software, or forecast and reanalysis data products) should be sent to the ECMWF Support Portal where they will be assigned to the appropriate team.