...
| Info |
|---|
Worked Example: We explain the model installation and the process of running of a forecast experiment on the ECMWF HPC facility (hpc2020 Atos).
It is important to note that the installation process on hpc2020 will not directly translate to alternative systems |
...
| Code Block | ||||
|---|---|---|---|---|
| ||||
git clone ssh://git@git.ecmwf.int/~dahaoifs/openifs-main-git_48r1.0.git |
| Note |
|---|
|
...
| Code Block | ||||
|---|---|---|---|---|
| ||||
export OIFS_HOME=$HOME/openifs/openifs_48R148r1.0/ |
The other model environment variables are
...
- This file can be located anywhere on your file system, however the recommended default location
is $OIFS_HOME/binscripts. - We provide a template for this configuration file in
$OIFS_HOME/binscripts/oifs-config.edit_me.sh. - You should edit this file and update the path set in variable
OIFS_HOMEwith your installation's path.
...
Starting the build process
The $OIFS_HOME/binscripts/openifs-test.sh script can be used to build the model and run initial tests.
...
| Code Block | ||||
|---|---|---|---|---|
| ||||
cd $OIFS_HOME ./binscripts/openifs-test.sh -cbt |
where
...
| Info |
|---|
By default, on hpc2020, OpenIFS will be built using the intel compilerIntel compiler (Intel 2021.2) . OpenIFS will also build with GNU, if this is required then the user should execute the following command
This will load the compiler environment for gnu GNU GCC 11.2. |
If everything has worked correctly then all tests should have passed and the script returns the following
...
An example forecast experiment has been prepared for OpenIFS 48r1. The experiment ID is is i4xc.
Extract the example forecast experiment i4xc.tgz into a folder in a location suitable for model experiments. This folder will be your experiment directory.
Example:
On the ECMWF hpc2020 our model installation $OIFS_HOME will be in $HOME/openifs_48r1.0 and for the experiment we extract the i4xc package to $PERM. The experiment directory shall therefore be $PERM/i4xc/2016092500.
...
| Code Block | ||||
|---|---|---|---|---|
| ||||
cd $PERM/i4xc/2016092500 cp ./ecmwf/fort.4 . cp ./ecmwf/wam_namelist . |
You will need to copy two three further scripts from the OpenIFS package into your experiment directory:
oifs-run:this is a generic run script which executes the binary model program file.exp-config.h:this is the experiment configuration file that determines settings for your experiment. It will be read by oifs-run.run.ecmwf-hpc2020.job:this is the wrapper script to submit non-interactive jobs on hpc2020
Copy both these files from $OIFS_HOME/bin scripts into your experiment directory.
| Code Block | ||||
|---|---|---|---|---|
| ||||
cd $PERM/i4xc/2016092500 cp $OIFS_HOME/binscripts/oifs-run . cp $OIFS_HOME/binscripts/exp-config.h . cp $OIFS_HOME/binscripts/run.ecmwf-hpc2020.job |
Determine experiment parameters
...
- A job wrapper script that is suitable for the locally available batch scheduler needs to be used to call
oifs-run. - We include an example job wrapper script
run.ecmwf-hpc2020.jobin$OIFS_HOME/binscripts, which is suitable for the ECMWF hpc2020 Atos HPC. This uses the SLURM batch job scheduler.- In section 3, this script is copied to the experiment directory because it needs to be located here, to run an experiment.
run.ecmwf-hpc2020.jobneeds to be edited with the following essential and optional changess- Intially
run.ecmwf-hpc2020.jobsets thePLATFORM_CFGvariable as follows
- Intially
...
- It is important to change
"/path/to/your/config/oifs-config.edit_me.sh"to the actual path for theoifs-config.edit_me.sh,e.g.,"$HOME/openifs/openifs_48R148r1.0/binscripts/oifs-config.edit_me.sh" - The default resources requested in
run.ecmwf-hpc2020.jobare 8 nodes on ECMWF hpc2020, with a total of 256 MPI tasks and 4 OpenMP threads. This can be changed as required. - For information, the
LAUNCHcommand for batch job submission is set to "srun" without any further options, because all required parallel environment settings are provided through the SLURM script headers.
- It is important to change
...