Versions Compared
Key
- This line was added.
- This line was removed.
- Formatting was changed.
The OpenIFS Data Hub is a facility to generate initial conditions for OpenIFS forecast model experiments. It is available only to users of the IFS or the OpenIFS model.
If you are interested in ECMWF datasets or need more general information about ECMWF data please follow this link.
Latest Updates
Tip |
---|
2023-05-16: The OpenIFS Data Hub is now online and available to users. Please see below regarding new URL and login credentials. |
What is the OpenIFS Data Hub
One of the most pressing needs OpenIFS users face is to create initial conditions to start a model experiment. Until now such initial experiment data had to be requested by contacting OpenIFS support via email.
The OpenIFS Data Hub allows OpenIFS and IFS users to produce their own initial experiment data through a web-based application that triggers an automatic data generation process on our servers. The initial data files are produced using the exact same method as if the OpenIFS team had created the data. At the end of the process the user is provided with a download web link.
The Data Hub removes therefore the need to email the OpenIFS team at ECMWF and provides one common system that is available to both ECMWF staff and external OpenIFS users everywhere. Any initial data types that are currently not yet served by the Data Hub can be requested, as before, from the OpenIFS team by emailing openifs-support@ecmwf.int.
After the initial public release of the OpenIFS Data Hub, following a pilot scheme phase, we will explore possibilities to expand this data facility with additional functionality, for instance by offering a wider range of data types or by providing access to OpenIFS training data. Feedback from users will be very important for this.
Getting started
The OpenIFS Data Hub can be accessed here:
Button Hyperlink | ||||||||
---|---|---|---|---|---|---|---|---|
|
You will need an ECMWF user account to access the Data Hub.
Info |
---|
Note: Your ECMWF login credentials are required to use the Data Hub. At this time your ECMWF username needs to be added manually to the user policy to work with the Data Hub. If you are a new user of the Data Hub and have access problems please contact openifs-support@ecmwf.int. If you have used the Data Hub before the migration to Bologna with bespoke login details, then you should now use your normal ECMWF login credentials which should already have been activated for the migrated Data Hub. You will no longer require the bespoke login details. |
After the initial user authentication you will reach the Data Hub's dashboard page which shows your data request history.
To submit a new request for initial data press the "New Request" button at the top of the page. This will open a new web page containing a data request form:
- Data Parameters:
- At present the Data Hub only produces initial experiment data. We expect that in future releases different data types can be selected here.
- Initial experiment data can be produced either from ERA5 reanalysis data or from historical data in the operational archive.
- Experiment Parameters:
- The start and end date determine the dates for which initial experiment data will be requested. If only one start date is required both dates need to be equal and in this case the value of the time increment between forecasts is ignored.
- For data generated from ERA5 reanalyses the earliest possible start date is 1940-Jan-01. The end date of the most recent available data is being continually updated and is typically 3 months prior to the present date. If recent start or end dates (e.g., within the past 3-4 months) are required then the user should verify first on the MARS Catalogue or on the Climate Data Store whether ERA5 data for the requested dates is currently available before submitting the request with the OpenIFS Data Hub. If no data is available then the data request in the OpenIFS Data Hub will be aborted.
- For data generated from the operational archive the earliest possible start date is 1979-Nov-01. During the pilot phase the end point of available data has been fixed to 2022-Jun-01.
- The forecast length determines the number of days for which boundary conditions are supplied (e.g. sea surface temperatures etc), commencing with the initial start date. This value must be between 24 and 2976 hours (i.e. between 1 and 124 days). The timestamp of the final requested data point must be at least 24 hours before the present day (for operational archive) or within the available valid data period (for ERA5).
- Model Parameters:
- Initial data for model cycles CY43R3 and CY48R1 can be produced for OpenIFS or the IFS model. Other model cycles before CY48R1 will not be supported.
- The initial data needs to have the grid type, horizontal resolution and vertical level numbers that are required for the experiment.
- Users can select between several Gaussian grid types. Please refer to the section on model grid and resolution in the OpenIFS User Guide for more information.
- The choice of vertical model levels is restricted to 60, 62, 91 or 137 levels to ensure availability for all a possible grid resolutions.
Once the data request form has been completed, the request can be submitted to the Hub by pressing the Submit button.
Once this button has been pressed, the request information is added to the queue and the data generation process can begin. The request history dashboard page is shown.
Some preliminary checking is done while entering the request data.
Info |
---|
If the dashboard page is not shown after pressing the submit button check your input fields for any errors. |
Further consistency checks will be carried out once the request has been accepted. Note that any user choices resulting in an unworkable data requests will be identified early in the process and the user request will be aborted.
A maximum data volume of 500,000 MB is permitted for a single request. Any requests exceeding this data volume will be aborted. For reference, an initial data request at the current operational grid resolution (Tco1279L137) for a single start date with 10 days forecast length has an (uncompressed) data volume of approximately 20,000 MB.
Examples for completing the request form
First example: You require initial data for a single start date, created from ERA5 reanalysis data.
- The field "Source for initial data" needs to display ERA5 reanalysis (which is the default value).
- Ensure that both date fields "First start date" and "Last start date" are set to the same date. When both dates are the same, any value selected for "Time increment between forecasts" will be ignored.
- Note that ERA5 data may not yet be available for the most recent months. If in doubt, check the MARS Catalogue prior to submitting your request.
- Select the experiment start time (default is 00:00 UTC) and the forecast length. The forecast length value determines the number of days for boundary conditions data (e.g. SSTs) in the ICMCL*INIT file.
- Select the model cycle and the model grid parameters for your request.
- Press "Submit".
Second example: You require initial data with multiple start dates, every 12 hours, created from historical data in the operational archive data.
- The field "Source for initial data" should now be set to Operational archive.
- Note that the default value for the start date is set to the most recent possible date for operational archive data.
- Ensure that the "First start date" and "Last start date" are set to the appropriate range of dates for your request; they must not be the same value for multiple start dates.
- In this case the value for "Time increment between forecasts" will be evaluated. For 12-hourly gaps between forecast start dates set this to 12.
- Complete all the other fields as in the first example.
- Press "Submit".
Checking the request progress
After submitting a request the dashboard page is displayed again showing the user's request history.
- A new request entry has appeared at the top of the request history.
A limit for open data requests (having status "new" or "in progress") applies per user and once this limit is reached no further requests can be submitted until some of the user's requests have completed or aborted. - At regular time intervals the system checks for new data requests. Once the request has been picked up its status indicator updates to "processing".
- Each request line entry has on the right side of the page a link to a corresponding Details page.
Clicking on this link will open another web page with information about the data request details and its generation progress.
On this Details Page for a selected request the following information is shown:
- The Details box in the upper left area shows the main request properties and its current status. After completion a download link for the data tarball will be displayed.
Note that the download link has an expiry time stamp. After this expiry time the download link will be no longer active, by this time the generated data is removed from the storage area and is not available for download any longer. - The Config box on the right side of the Details Page displays the parameters that were selected previously in the input form.
- The Changelog box displays the status history of the request with the current status on top. Any potential errors or other reasons for aborting the request will be shown here.
Panel | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|
| ||||||||||