...
| Info |
|---|
Worked Example: We explain the model installation and the process of running of a forecast experiment on the ECMWF Atos Sequana XH2000 HPC facility (hpc2020).
It is important to note that the installation process on hpc2020 will not directly translate to alternative systems. To address this we present details about a docker install in the last section of this guide. |
Access requirements
OpenIFS is licensed software and using the model requires your affiliation with an institute that has signed the ECMWF OpenIFS software licence agreement.
...
If you are a new model user you should in the first instance contact OpenIFS support (by emailing openifs-support@ecmwf.int) requesting access to OpenIFS by providing the following information: your full name, your affiliation and related institutional email address, and your ECMWF account user name. OpenIFS staff will then activate OpenIFS user policies for your personal ECMWF account. In some cases we may need to request additional verification of your affiliation with your licensed institute prior to giving you access.
Extract the OpenIFS package
Create your local installation of OpenIFS 48R1.1 by downloading and extracting the tarball containing the model source package.
...
| Note |
|---|
|
Set up the platform configuration file
The OpenIFS model requires a number of Linux environment variables to be set for both installation and runs. These are referred to as global environment variables.
...
| Info |
|---|
|
Build OpenIFS
In the next step the model binary executable (and other helper programs) will be built.
OpenIFS build system
In contrast to earlier model versions, the building of OpenIFS 48r1 is no longer based on the FCM configuration manager; instead, the build system for OpenIFS is now based on the same ecbuild / CMake system used for the ECMWF IFS model.
Similar to the IFS, OpenIFS 48r1 also employs a software bundle which, as part of the build process, automatically finds, downloads and installs many of the required software packages, e.g., ecbuild, ecCodes, metkit, etc. Hence, a separate installation of these libraries is no longer required as they have now become part of the OpenIFS distribution.
Starting the build process
The $OIFS_HOME/scripts/build_test/openifs-test.sh script can be used to build the model and run initial tests.
...
| Note |
|---|
100% pass with |
Set up a forecast experiment
An example forecast experiment has been prepared for OpenIFS 48r1. The experiment ID is ab2a.
...
| Code Block | ||||
|---|---|---|---|---|
| ||||
cd $OIFS_EXPT/ab2a/2016092500 cp $OIFS_HOME/scripts/exp_3d/oifs-run . cp $OIFS_HOME/scripts/exp_3d/exp-config.h . cp $OIFS_HOME/scripts/exp_3d/run.ecmwf-hpc2020.job . |
Determine experiment parameters
Namelist:
- You can edit the atmospheric model namelist file fort.4. It contains Fortran namelists which control model settings and switches.
- An important switch to edit is in namelist NAMRIP the variable CSTOP. Set this to the desired length of the forecast experiment.
- Experiment ab2a can be run for up to 144 hours (6 days) by setting
CSTOP='h144'.
...
| Info | ||
|---|---|---|
| ||
|
Running the experiment
After all optional edits to the namelists (fort.4) and to the experiment configuration file (exp-config.h) have been completed the model run can be started.
Depending on the available hardware experiments can either be run interactively or as a batch job.
Running a batch job
This method is the preferred way to run OpenIFS, as it is more efficient and it allows more flexibility in using the available hardware resources.
...
| Info |
|---|
The job wrapper script will read the exp-config.h file and adopt the selected values. The exceptions are LAUNCH, which is set to "srun" for batch jobs, and OIFS_NPROC & OIFS_NTHREAD for which values from the batch job headers are used. The job wrapper script modifies the exp-config.h file accordingly prior to calling the oifs-run script. |
Running interactively
On the ECMWF hpc2020, running the model script interactively should be fine for lower grid resolutions up to T255L91.
...
| Code Block | ||||
|---|---|---|---|---|
| ||||
# run interactively: cd $OIFS_EXPT/ab2a/2016092500 ./oifs-run |
Postprocessing
If in the exp-config.h file the OIFS_PPROC variable has been set to true (or if the --pproc command line parameter was used) then the model output in the experiment directory is further processed after completing the model run.
...
This postprocessing is required if the Metview Python script is to be used later to visualise the model output.
Plotting of model output
Here we describe in a brief summary how plots from the model results can be generated. This permits a first-order sanity check whether the model results look sensible.
...
- We describe the process on the example of the ECMWF Linux Virtual Desktop (VDI)
- In the VDI, open a terminal, log into the hpc2020 with command:
ssh hpc-login - In the terminal start the Jupyter session on an interactive node, using the command:
ecinteractive -j - After the interactive node has started you will be given a weblink to connect to the Jupyterlab session ("To manually re-attach go to <weblink>").
- Open a web browser (e.g. Chrome) inside the VDI and paste the weblink into the browser's URL address field; this will connect to the Jupyter session.
- In the file explorer, on the left side of the Jupyter window, navigate to the folder
$PERM/mv/ipynb/and select Notebooksingle.ipynb - Open this Notebook by double-clicking in the explorer window.
- Once it has opened, run all its cells in sequence (e.g. use the command "Run All Cells" in menu "Run").
- This will generate a series of plots from the model output which are displayed inside the Notebook.
- Optional: After completing the Jupyter session it is good practice to release the reserved interactive node using this command in the terminal window:
ecinteractive -p hpc -kand confirm cancellation of the job; if this is not done the interactive job will timeout after 12 hours.
Requirements
This section provides further details about software requirements for OpenIFS.
...