This page contains a guide for building and using OpenIFS 48r1 at model version 48R1.1.

We describe how to build the model and carry out a few basic tests, how to run an example forecast experiment, for which initial experiment data is made available, and how to visualise the model output by creating plots.

Worked Example:

We explain the model installation and the process of running of a forecast experiment on the ECMWF Atos Sequana XH2000 HPC facility (hpc2020)

  • All model sources, data files and configuration scripts that are required for the worked example experiment described in this guide can be found on the ECMWF web site here: https://sites.ecmwf.int/openifs/openifs-data/ (only registered OpenIFS users will be given access)
  • For users with access to the ECMWF HPC facility we provide an internal filepath location for the data packages (without the model source code). 
  • In our example the model will be installed into $HOME/openifs-48r1/ as a default location.
  • The example forecast experiment will be set up in directory $OIFS_EXPT/ab2a/

It is important to note that the installation process on hpc2020 will not directly translate to alternative systems. To address this we present details about a docker install in the last section of this guide.

1. Access requirements

OpenIFS is licensed software and using the model requires your affiliation with an institute that has signed the ECMWF OpenIFS software licence agreement.

In order to access the model sources and other required data packages, you need to have a personal ECMWF account which you can create on the ECMWF web site or by clicking here.

If you are a new user of OpenIFS 48r1 you should in the first instance contact OpenIFS support requesting access to OpenIFS by providing the following information:

  • your full name,
  • your affiliation and related institutional email address,
  • and the username of your personal ECMWF web user account.

OpenIFS staff will then activate OpenIFS user policies for your personal ECMWF web user account. In some cases we may need to request additional verification of your affiliation with your licensed institute prior to giving you access. 

The OpenIFS model is available as tarfiles from an ECMWF download site along with example initial experiment data and additional software. 

In order to access the download site prospective model users need to:

  (1) be affiliated with an institution that has signed the OpenIFS licence agreement,
  (2) have created their own personal ECMWF web user account,
  (3) and contact OpenIFS support to have their account added to the OpenIFS user group. 

Note:  Access to the download site is only granted after the personal ECMWF web user account was added to the OpenIFS user group. If you have previously used OpenIFS 43r3 and wish to upgrade to using OpenIFS 48r1, then you need to contact OpenIFS support with your ECMWF web user account to get access. 

2. Extract the OpenIFS package

Create your local installation of OpenIFS 48r1 by downloading and extracting the tarball containing the model source package.

The tarball can be downloaded from this web site: https://sites.ecmwf.int/openifs/openifs-data/src/48r1/. It is named  openifs-48r1-YYYYMMDD.tar.gz, where YYYY, MM, and DD stand for the year, month and day of the most recently created release. The CHANGES file in the source package describes the developments between releases.

Please note that access to this web site is restricted to registered OpenIFS users, i.e. your personal ECMWF account must have previously been added to the OpenIFS user group.

In the example below we assume that the OpenIFS model shall be installed in the user's HOME directory: 

# download the openifs-48r1-20240209.tar.gz model package and copy the tarball to your home directory:
cp openifs-48r1-20240209.tar.gz $HOME
tar -xvzf openifs-48r1-20240209.tar.gz
  • You will need approximately 4 GB of disk space for the model sources, the bundle packages, and the built model binaries. 
  • Please note that in future access to the OpenIFS source package will be provided through a git repository. Therefore the provision of the package as a tarball is only a temporary arrangement. 

If you are experiencing difficulties with extracting the tarball then you may want to verify the integrity of the downloaded tarball file. If the md5sum utility is available on your system, you can verify the file's MD5 checksum.

To do this please enter the command:  md5sum openifs-48r1*tar.gz. This will produce an alphanumeric checksum string which should be identical with the one in the file openifs-48r1*.md5. If they do not match then your local tarball file might be corrupted (most likely due to an incomplete download) and you should try to download it again.

2.1. Set up the platform configuration file

The OpenIFS model requires a number of Linux environment variables to be set for both installation and runs. These are referred to as global environment variables.

The most important environment variable is OIFS_HOME, which is required by all scripts used by the model. 

 OIFS_HOME describes the location of the OpenIFS model installation and is general the path where the git repository was extracted.  For example, if you clone the repository into your $HOME directory then you should set:

export OIFS_HOME=$HOME/openifs-48r1/

The other model environment variables are

  • OIFS_CYCLE - describes the model cycle (e.g. 48r1) for which this configuration file can be used - this should not be edited 
  • OIFS_EXPT - path to the location of the openifs experiments. Setting this is a convenience for the OpenIFS 3D model but is a requirement for the SCM experiments
  • OIFS_ARCHif available on for a system, this variable describes the location of the arch directory,  which provides specific information about the system and compiler, e.g. $OIFS_HOME/arch/ecmwf/hpc2020/gnu
    • Such a directory is not always required, i.e., if a system has all the appropriate libraries installed. If this is the case OIFS_ARCH can be set to an empty string, i.e., OIFS_ARCH="" 
  • OIFS_DATA_DIR - describes the location of climatological input files that are required to run OpenIFS. These have been installed on the ECMWF hpc2020 in a central and accessible location under /perm/openifs/oifs_data and the information is organised by model cycle. 

If you do not have access to the ECMWF hpc2020 file system, or if you wish to install the climatological input files in a local directory of your choice, then you can download the required data from this site: https://sites.ecmwf.int/openifs/openifs-data/ifsdata/48r1/

As a minimum you will require the packages ifsdata_rtables_48r1.tgz and ifsdata_climatology_48r1.tgz. You will also need to download the package for your selected horizontal grid resolution, which in the case of our worked example on this page is for a T255 grid, and therefore you will at the very least need the package ifsdata_48r1_climate.v020_255.tgz. Installing the packages for all supported horizontal grids will require a lot of disk space and is therefore not recommended.

Download and extract the files in these tarballs into the same directory in your chosen location, and the filepath of this directory should then be set to variable OIFS_DATA_DIR

  • OIFS_EXEC - describes the location and file name for the OpenIFS 3D executable, e.g. $OIFS_HOME/build/bin/ifsMASTER.DP , which is a double-precision executable for the 3D model

All the global environment variables described above, as well as some other variables that are dependent on the above, are defined in the Platform configuration file, which needs to be modified for your local OpenIFS installation. 

  • This file can be located anywhere on your file system, however the recommended default location is $OIFS_HOME.
  • We provide $OIFS_HOME/oifs-config.edit_me.sh, which is a commented template of this configuration file.
  • You should edit this file and update the path set in variable OIFS_HOME with your installation's path.

Once edited the platform configuration file is loaded using the following command:

source /path/to/file/location/oifs-config.edit_me.sh
# using our installation example:
# source $HOME/openifs-48r1/oifs-config.edit_me.sh

We recommend to include this command in your Linux shell startup configuration (e.g. in .bashrc). 

The above command should also be include in any batch job scripts that are intended to run OpenIFS (described in Section 3)

$OIFS_HOME/oifs-config.edit_me.sh also contains a environment variables that relate to the Single-Column Model (SCM). These are discussed in Getting started with the SCM

3.  Build OpenIFS

In this section we describe how the model binary executable (and other helper programs) will be built.

3.1. OpenIFS build system

In contrast to earlier model versions, the building of OpenIFS 48r1 is no longer based on the FCM configuration manager; instead, the build system for OpenIFS is now based on the same ecbuild / CMake system used for the ECMWF IFS model.

Similar to the IFS, OpenIFS 48r1 also employs a software bundle which, as part of the build process, automatically finds, downloads and installs many of the required software packages, e.g., ecbuild, ecCodes, metkit, etc. Hence, a separate installation of these libraries is no longer required as they have now become part of the OpenIFS distribution. 

3.2. Starting the build process

The  $OIFS_HOME/scripts/build_test/openifs-test.sh  script can be used to build the model and run initial tests. 

  • The script requires key environment variables, such as $OIFS_HOME, to be assigned. Make sure you have sourced the platform configuration script first.
  • The usage of the script is shown with the command:  openifs-test.sh -h

Run the build process and the tests using the following commands (assumes the platform configuration file has been sourced) :

cd $OIFS_HOME
$OIFS_TEST/openifs-test.sh -cbt

where

$OIFS_TEST is defined in the platform configuration file (oifs-config.edit_me.sh) as $OIFS_HOME/scripts/build_test 

and the command line options are

-c  creates source directory in $OIFS_HOME, which is used to bring all the sources, e.g. ifs-source, ifs-test and any packages in the bundle (ecbuild, eccodes etc.), together in preparation for the build

-b  builds the source. This step creates the directory build in $OIFS_HOME, which is used to build and store

  • the OpenIFS double and single precision master executables (ifsMASTER.DP and ifsMASTER.SP, respectively) , which are used to run 3-D OpenIFS. The executables are located in  $OIFS_HOME/build/bin.
    • The location and name of the executable for OpenIFS is defined in the platform configuration file (oifs-config.edit_me.sh) as $OIFS_EXEC.
  • the double and single precision Single Column Model (SCM) executables (MASTER_scm.DP and MASTER_scm.SP, respectively), which are used to run the SCM derived from OpenIFS. The executables are located in  $OIFS_HOME/build/bin.
    • The location and name of the executable for the SCM is defined in the platform configuration file (oifs-config.edit_me.sh) as $SCM_EXEC.

-t will run the ifs-test t21 tests, which comprise of

  • 21 3-D OpenIFS forecast-only tests with and without chemistry
  • 1 SCM test (based on TWP-ICE)

-j is an optional argument that allows the user to define the number of threads used to create and build openifs, e.g.

$OIFS_TEST/openifs-test.sh -cbt -j 16

which will perform the create and build step using 16 threads

  • If -j is not supplied as a command line argument, as in the above argument, the number of threads will default to the IFS build options for 48r1, i.e. 
    • 64 threads for ATOS
    • 8 threads for all other systems
  • If -j argument is present, then the command line number of threads takes precedence over the defaults for any system 
    • The default of 8 threads can be increased for larger systems. Such an increase may speed up the build. For advice on this please liaise with the system administrator.
    • It is important to note that 8 threads can be problematic for older and lower spec systems. For example, it will not work with a 4 core intel m3. In this case reduce the number of threads to 2 for example. 

In addition to the command line options defined above (-cbt) , openifs-test.sh invokes some default ecbuild options and allows a user to add ecbuild options from command line using the -- notation, e.g., $OIFS_TEST/openifs-test.sh -cbt --without-single-precision. Openifs-test - build options describes some of these options.


If everything has worked correctly with $OIFS_TEST/openifs-test.sh -cbtthen all tests should have passed and the script returns the following

[INFO]: Good news - ctest has passed
        openifs is ready for experiment and SCM testing
----------------------------------------------------------------
END ifstest on OpenIFS build

100% pass with $OIFS_TEST/openifs-test.sh -cbt shows that the low resolution (t21) ifs-test cases can run to completion on the chosen system. These tests do not check bit comparibility with known good output. If this is a requirement, e.g., if a user makes a code change and needs to test whether the code has led to unexpected behaviour in the code, then Openifs - bit identical testing page describes how to set this up with openifs-test.


Next Steps:

This page has shown you how to download, extract, build and test the OpenIFS 3D global model on your local computing system. The next step will be to carry out a first global forecast model run for which we have prepared an example case study experiment. To do this please continue on the page Setting up and running a forecast experiment