This page contains a guide for building and using OpenIFS 48r1 at model version 48R1.1 in a linux based container using Docker.

While this page focuses on the containerised installation, the details presented can be used as a recipe for the underlying dependencies required to install and run OpenIFS 48r1 on a linux system.

If you have access to ATOS hpc2020 and are planning to run OpenIFS on that system please refer to OpenIFS 48r1 - Getting Started (ATOS hpc2020).

Contents:

Creation of a docker build directory

The first step is to make a directory from which the OpenIFS container can be created and cd into that directory, e.g.

mkdir <openifs-docker-build-dir>
cd <openifs-docker-build-dir>

Once inside the <openifs-docker-build-dir> clone or branch from the git repository with the OpenIFS release package, e.g.

git clone --single-branch --branch release ssh://git@git.ecmwf.int/oifs/openifs-48r1.1.git

Please note that currently access to this repository is restricted to a limited number of users only.
It is important that the OpenIFS package in the docker build directory is a fresh clone/branch, i.e., not built on your local system.

Once OpenIFS 48r1 has been extracted to your <openifs-docker-build-dir> , then copy the Dockerfile from the release to <openifs-docker-build-dir>

cp  <openifs-docker-build-dir>/openifs-48r1.1/scripts/docker/gcc-docker-48r1.1/Dockerfile <openifs-docker-build-dir>/.

This cp step is important because the Dockerfile and the OpenIFS release, i.e., openifs-48r1.1, need to be in the same directory to build the container (see the openifs-48r1.1/scripts/docker/gcc-docker-48r1.1/Dockerfile)

Build the OpenIFS docker image

The following command builds the docker image

docker build --tag "openifs-48r1.1" .

Note the trailing '.' to build in the current dir, which is required.

This command runs the build process for the OpenIFS image using gcc:11.2.0-bullseye as the base image, which is a debian Linux distribution with gcc 11.2 (at the time of writing, similar to the gcc on the ATOS). After downloading the image, the Dockerfile installs the following software

    apt install -y git && \
    apt install -y cmake && \
    apt install -y python3 python3-ruamel.yaml python3-yaml python3-venv && \
    apt install -y libomp-dev && \
    apt install -y libboost-dev libboost-date-time-dev libboost-filesystem-dev libboost-serialization-dev libboost-program-options-dev&& \
    apt install -y netcdf-bin libnetcdf-dev libnetcdff-dev && \
    apt install -y libatlas-base-dev && \
    apt install -y liblapack-dev && \
    apt install -y libeigen3-dev && \
    apt install -y bison && \ 
    apt install -y flex && \
    apt install -y vim && \
    apt install -y wget bc

Once these packages are installed, open-mpi is downloaded and built, the openifs user is created and the openifs-48r1.1 directory is copied to the image.

On a Macbook pro (M1) the initial build of the container, takes about 5 minutes. If you decide to change the Dockerfile following a build, you can just execute the same command and, depending on what you change, subsequent builds can be a lot quicker.

At the time of writing open-mpi is downloaded and built as part of the image creation. This is quite slow, so in the future we will investigate the use of a standard library
If a fresh build is required but an image has already been built, then execute the following command, which is the same as above but with no-cache

docker build --no-cache --tag "openifs-48r1.1" .

The tag name does not need to be provided but it is useful for identifying images.

Running OpenIFS-test using the docker image

Once a build has completed, the image can be checked by typing docker images , e.g.,

docker images

REPOSITORY          TAG       IMAGE ID       CREATED       SIZE
openifs-48r1.1      latest    f72ea92f010f   5 hours ago   2.15GB

And the container can be run using the following

docker run -it 'openifs-48r1.1'

This command will open an interactive session in the new container, in which the entry directory is the openifs-48r1.1 directory, e.g.

docker run -it 'openifs-48r1.1'

OpenIFS environment variables are:
------------------------------------------------------
OIFS_CYCLE=48r1
OIFS_DATA_DIR=/perm/openifs/oifs_data/48r1/48r1
OIFS_HOME=/home/openifs/openifs-48r1.1
OIFS_LOGFILE=/home/openifs/openifs-48r1.1/oifs_test_log.txt

openifs@d1bd89ccc47f:~/openifs-48r1.1$ ls
CHANGES  COPYING  NOTICE  arch        ifs-source  oifs-config.edit_me.sh  scripts
CITE     LICENSE  README  bundle.yml  ifs-test    openifs-bundle

In the above code block the openifs config (oifs-config.edit_me.sh) is sourced automatically so the container has the correct environment for OpenIFS

Once inside the container, the openifs-test.sh script can be run, e.g.,

openifs@d1bd89ccc47f:~/openifs-48r1.1$ ./scripts/openifs-test.sh -cbt -s docker

where :

-c will clean the directory (not necessary for a fresh image or new container)

-b will configure the bundle and build OpenIFS, i.e.,

the OpenIFS double and single precision master executables (ifsMASTER.DP and ifsMASTER.SP, respectively) , which are used to run 3-D OpenIFS. The executables are located in $OIFS_HOME/build/bin.
the double and single precision Single Column Model (SCM) executables (MASTER_scm.DP and MASTER_scm.SP, respectively), which are used to run the SCM derived from OpenIFS. The executables are located in $OIFS_HOME/build/bin.

-t will run the ifs-test

21 3-D OpenIFS forecast-only tests with and without chemistry
1 SCM test (based on TWP-ICE)

-s docker defines the system as docker, which makes the script use the correct build commands

On a Macbook pro (M1) the build of OpenIFS, takes about 10 minutes, while successful completion of ifstest takes just over 3 minutes.

Basic docker commands

Check for existing containers

From a terminal on the host system, it is possible to list running and exited containers with the following

List running containers

$ docker ps
CONTAINER ID   IMAGE            COMMAND   CREATED        STATUS         PORTS     NAMES
d1bd89ccc47f   openifs-48r1.1   "bash"    15 hours ago   Up 5 seconds             beautiful_pasteur

If no container is running on your system, then only CAPITAL headings are returned with docker ps

List all containers (running and exited)

$ docker ps -a
CONTAINER ID   IMAGE               COMMAND       CREATED        STATUS                     PORTS     NAMES
d1bd89ccc47f   openifs-48r1.1      "bash"        15 hours ago   Exited (0) 4 seconds ago             beautiful_pasteur

Notice that the STATUS is Exited, rather than Up, as in (1)

Start an existing container

If docker ps -a shows an exited container it can be restarted using the following

$ docker start -i <Container ID> 
# e.g. 
$ docker ps -a
CONTAINER ID   IMAGE               COMMAND       CREATED        STATUS                     PORTS     NAMES
d1bd89ccc47f   openifs-48r1.1      "bash"        15 hours ago   Exited (0) 4 seconds ago             beautiful_pasteur
$ docker start -i  d1bd89ccc47f

This will restart the container but it is important to note that there will be no data or changes retained from when the container has been used in the past.

Exiting the container

The container can be exited by typing exit from the active container.
A container can be stopped from a terminal on the host system

Once exited all data and changes made within the container will be lost. Hence, if necessary, it is important to
- Push any code changes back to a repo
- If data is required either set-up a data volume or copy the data back to a local machine, using docker cp

Start an existing container
Executing docker run will always create a new container, which will be retained if the run command does not include the --rm. If a container is not removed when exited, then it can be restarted