...
Introduction
OpenIFS is an ECMWF led project which provides an easy-to-use, exportable version of the IFS system in use at ECMWF for operational weather forecasting. The project aims to develop and promote research, teaching and training on numerical weather prediction (NWP) and NWP-related topics with academic and research institutions.
In this tutorial, OpenIFS has been pre-installed and pre-compiled for you. If you want to learn more about how OpenIFS can be downloaded, configured and installed, please see the OpenIFS website at: http://software.ecmwf.int/oifs/.
In this tutorial...
What you will do:
- Connect to the ECMWF HPC Facility
- Explore installed OpenIFS files
- Run T21 model forecast
- Verify the model works in serial (1 process) and parallel (2 tasks & 2 threads)
- Carry out acceptance test
- Use the model namelist to change settings
- Explore model output data (GRIB)
At the end of this tutorial, you will:
- have learnt about how OpenIFS is installed and organised,
- have run it at T21 resolution and know how to run it serially (1 process) and in parallel with both MPI & OpenMP,
- know how to use grib tools to look at model output.
Connecting to the ECMWF HPC Facilities
In this tutorial, and throughout the entire workshop, you will use a Cray XC40 which is part of the ECMWF high-performance computing facility (HPCF).
...
On the ECMWF HPCF an interactive session will last for 48 hrs by default, unless the walltime is specified using an additional directive.
OpenIFS directories
In this section we:
- Set the OpenIFS environment
- Examine the OpenIFS installation
...
- OpenIFS has been precompiled on the HPCF.
- All source code has been removed due to licensing restrictions.
- OpenIFS builds 'out-of-source'; this means object (
.o
) files and executables (binary files) are not mixed with the source code. - The
README
file contains information about software requirements, setting up the local compilation environment, and where to get help and support.
OpenIFS T21 test forecasts
In this section of the tutorial, we will run the pre-compiled OpenIFS model on a simple T21 forecast.
...
Whenever the model fails, it will produce this traceback (controlled by DR_HOOK=1 in the job file).
Single process test
Panel | ||||
---|---|---|---|---|
| ||||
Copy the file
|
...
This test runs only 6 timesteps.
Model output
The model writes its output to a several files.
...
Panel | ||||
---|---|---|---|---|
| ||||
Look at the output from the model in the Look at the grib output files using the grib commands. |
Parallel runs
These next short tests verify the model works correctly with either OpenMP parallel threading, MPI tasks and both and follow on from the serial tests above.
...
Look in the NODE_001.01
output file for the line: "NPROC = 4
" to verify that two MPI tasks was used.
Mixed mode: OpenMP/MPI
The model can also be set to use NPROC=4
and NTHREADS=4
to use a total of 4 processes. However, this would require a computer with at least 16 cores.
Acceptance testing
The final step is to check the model is producing the numerical answers within acceptable limits, even if it runs the short tests above without failing. OpenIFS includes code that will compute internal statistical norms and compare against numbers supplied by ECMWF. The file: ref_021_0144
in the t21test
directory contains statistical norms computed by the model run at ECMWF.
...
As long as the model reports 'calculations are correct' and the error is less than a few percent then the model is behaving satisfactorily in your compilation and run environment.
How to control model output
In this section, the main NAMELIST variables that control the output of the model are presented.
If you have time, try changing the variables, run the model and using the grib_ls and grib_dump commands to view the output grib file contents.
How to control output frequency
The namelist variables that determine the output from the model as it runs are:
Namelist : NAMCT0
NFPOS
- this should be set =2 in order to turn on model output and diagnostics.
...
NPOSTS & NHISTS - these are integer arrays that control the write times of the history files. They can be used for non-regular output intervals.
Examples
Regular output at fixed timesteps
NFRHIS=4,
NFRPOS=4,
NPOSTS=0,
NHISTS=0,
...
Note |
---|
For this to work correctly, |
Non-regular output
NFRHIS=1,
NFRPOS=1,
NHISTS(0)=3,
NHISTS(1:3)=0,-3,-9,
NPOSTS(0)=-3,
NPOSTS(1:3)=0,-3,-9,
...
In this example, the model will write 3 separate output files at the first timestep (0hrs), 3hrs and 9hrs and then no more regardless of how long the model runs for.
How to change the output variables and post-processing
The namelist NAMFPC
is the main namelist for the post-processing. Variables in this list can be sensitive to changes as many combinations are possible but not all work.
Model level output
To control model level output the following namelist variables (in NAMFPC) are used:
...
Panel | ||||
---|---|---|---|---|
| ||||
Using the information above, try:
|
At the end of the session
Note |
---|
Do not forget to close your interactive ppn session on the HPCF using the exit command. |
...