Set your environment

If this is the first time compiling OpenIFS, the first step is to setup the environment for compilation. A successful compilation relies on correctly specifying the location of system and user libraries.

Find the file oifs-config.editme.sh and make a copy to edit using your favourite editor:

% cd oifs43r3v1
% cp oifs-config.editme.sh  oifs-config.me.sh
#  .. then edit oifs-config.me.sh 

Once you have correctly set the environment file, 'source' it to set the environment variables:

% source ./oifs-config.me.sh
Environment for OpenIFS: CY43R3
OpenIFS home directory: /home/users/auser
OpenIFS top-level data directory: /home/users/auser/ecmwf/ifsdata
OpenIFS initial data directory : <editme>
OpenIFS GRIB directory: /home/users/auser/ecmwf/eccodes
OpenIFS netCDF directory: /usr/local/lib64
OpenIFS default compiler and build set to: gnu opt
Added OpenIFS FCM and GRIB to user path
Creating command aliases for OpenIFS: omake, omakenew, oenv

FCM

OpenIFS uses software called 'FCM' to handle the compilation of the code. FCM is distributed in the tarball downloaded from the ftp site.

Before trying to compile, make sure the FCM directory is added to your PATH environment variable so the fcm command can be found (if you already have fcm available on your system, we still recommend using the distributed version):

export PATH=$PATH:$HOME/ecmwf/oifs/fcm/bin

Build configuration

This is a brief introduction on compiling the OpenIFS model. For a more detailed description of the build system for OpenIFS and the different ways to configure it, see HowTo customize the OpenIFS build.

Before compiling the model, the build configuration must first be set by several environment variables:

OIFS_COMP - This sets the choice of compiler. The default is 'gnu' which means the gfortran/gcc compilers will be used.

OIFS_BUILD - This sets the type of build.

These environment variables directly correspond to the names of the FCM configuration files in the make/cfg/ directory in the source. Please see this directory for the choices provided.

Build types: OIFS_BUILD sets the type of build and there will be one FCM configuration file for each type. Build types provided are:

• opt - Recommended optimized compile settings for this compiler. Tested to provide best performance for this platform & compiler.
• noopt - Debugging options. No optimization set for all of the code. Suitable for use with a debugger. OpenMP is disabled.
• nansC - Debugging options with extra checks. Array bound checking enabled and variables initialized to NAN (not-a-number). Use of a variable set to NaN will trigger a floating point exception. This option is useful for debugging purposes but will run the slowest.

e.g.

export OIFS_COMP=intel
export OIFS_BUILD=opt

means using the Intel ifort compiler and the model will be compiled with full optimization compiler settings. So the FCM build system will expect to find a file:

make/cfg/intel-opt.cfgfcm

Setting:

export OIFS_COMP=gnu
export OIFS_BUILD=noopt

means FCM will expect to find a file:

make/cfg/gnu-noopt.cfgfcm

If using GNU on a Mac, please see MacOS X known issues.

Building for other systems/environments

As the environment variables refer to the corresponding filename in the make/cfg  directory it is straightforward to create a build configuration for other environments. Take a copy of an existing file and modify it as necessary. It's also possible to alter compile options by the use of additional environment variables (see below). Which approach you use depends on your personal preference. Be aware however that the optimization compiler flags in the provided configuration files are the recommended ones. It is impossible to test all combinations of compiler flags so the model may become unstable or performance may reduce if higher optimizations are tried.

Setting the location of libraries

The model requires a number of libraries (see 5.2 Prerequisites) to compile. As well as the GRIB library (either ECMWF's ecCodes or grib_api libraries), the LAPACK & BLAS numerical libraries are required. An MPI library is also required but this is assumed to be provided by compiler wrappers provided by the MPI implementation (e.g. the command mpif90 rather than gfortran).

To set the location of the GRIB & LAPACK libraries, two other environment variables need to be defined as the defaults in the FCM configuration files will most likely not be appropriate.

OIFS_GRIB_DIR - Defines the location of the GRIB library. For more details see '5.3 Install GRIB library for OpenIFS'.

OIFS_LAPACK_LIB - Defines the location of the directory containing the liblapack.a and libblas.a files. This varies from system to system but typically /usr/local/lib.

If you do not have LAPACK installed on your system or find it does not work, OpenIFS comes with all the necessary LAPACK code to run. It is disabled by default, contact openifs-support@ecmwf.int for instructions on how to enable it.

e.g.

export OIFS_GRIB_DIR=$HOME/ecmwf/eccodes_gcc
export OIFS_LAPACK_LIB="-L/usr/local/lib -llapack -lblas"

Or if you prefer you can either edit the default value in the appropriate FCM configuration file in make/cfg directory or make a copy of the supplied configuration file (.cfg) and use that.

$OIFS_GRIB_LIB{?} = -L$OIFS_GRIB_DIR/lib -lgrib_api_f90 -lgrib_api

$OIFS_GRIB_LIB{?} = -L$OIFS_GRIB_DIR/lib -leccodes_f90 -leccodes

export OIFS_GRIB_LIB="-L\$OIFS_GRIB_DIR/lib -leccodes_f90 -leccodes"

Compiling OpenIFS

Once the environment variables are set, OpenIFS can be compiled (make sure you have edited your PATH environment variable to add the FCM installation bin directory). Assuming you are in the directory where the OpenIFS source code was unpacked:

cd make
fcm make -f oifs.cfgfcm

The command 'fcm make' starts the compilation. The -f option specifies the location of the master configuration file for OpenIFS. For more information about fcm command options, try: fcm --help .

This generates output similar to:

[init] make                # 2012-12-20 19:59:59Z
[init] make config-parse   # 2012-12-20 19:59:59Z
[info] config-file=...../make/oifs.cfg
[info] config-file= - ...../make/cfg/gnu-opt.cfg
[done] make config-parse   # 0.3s
[init] make dest-init      # 2012-12-20 19:59:59Z
[info] mode=new
[done] make dest-init      # 0.3s
[init] make build          # 2012-12-20 20:00:00Z
[info] sources: total=2194, analysed=2194, elapsed-time=11.7s, total-time=32.2s
[info] target-tree-analysis: elapsed-time=9.8s
[info] compile   targets: modified=2073, unchanged=0, total-time=1095.5s
[info] compile+  targets: modified=628, unchanged=0, total-time=1.6s
[info] ext-iface targets: modified=1247, unchanged=0, total-time=21.9s
[info] install   targets: modified=120, unchanged=0, total-time=0.3s
[info] link      targets: modified=1, unchanged=0, total-time=18.3s
[info] TOTAL     targets: modified=4069, unchanged=0, elapsed-time=316.3s
[done] make build          # 329.1s
[done] make                # 329.8s

Parallel compilation

If you have a multicore processor available, the -j <nthreads> option to 'fcm make' can be used to speed up the compilation of the code.

% fcm make -j 4 -f oifs.fcm	

Tests show that compilation time reduces with a diminishing amount as the number of threads is increased.

Verbose compilation

The default output of FCM is quite terse. To get more verbose output use the '-v' option.  e.g.

fcm make -v -j 4 -f oifs.fcm

will produce a more detailed report and use 4 processes to analyse and build the model code in parallel.

Successful compilation

After a successful compilation, the OpenIFS executable, master.exe, can be found in the 'make/gnu-opt/oifs/bin' subdirectory (or whichever compiler you choose):

% ls
cfg  gnu-opt
% ls gnu-opt/oifs
bin   include   o
% ls gnu-opt/oifs/bin
master.exe

The 'gnu-opt' subdirectory is generated by FCM using the values of the OIFS_COMP & OIFS_BUILD environment variables. The oifs/include directory contains all the files referenced in the code by a '#include' statement. This includes the Fortran interface block files which are auto-generated by FCM and end in .intfb.h. The oifs/o directory contains all the compiled object files. The OpenIFS build does not place any additional files in the src directory (known as building 'out of source').

Compilation problems

If you get the message:

fcm make -f oifs.cfgfcm
/bin/ksh: fcm: not found [No such file or directory]

you need to add the 'bin' directory of the FCM installation to your PATH environment variable. FCM is supplied with OpenIFS and is in the directory 'fcm'.