Set your environment

First time compilation: oifs-config

If this is the first time compiling OpenIFS, the first step is to setup the environment for compilation. A successful compilation relies on correctly specifying the location of some system and user libraries (e.g. netCDF and ecCodes).

Find the file oifs-config.editme.sh and make a copy to edit using your favourite editor:

% cd oifs43r3v1
% cp oifs-config.editme.sh  oifs-config.sh
#  .. then edit oifs-config.me.sh 

See the comments in the 'oifs-config.sh' file for more information about the variables (or see below).

Once you have correctly set the environment file, 'source' it to set the environment variables:

% source ./oifs-config.me.sh
Environment for OpenIFS: CY43R3
OpenIFS home directory: /home/users/auser
OpenIFS top-level data directory: /home/users/auser/ecmwf/ifsdata
OpenIFS initial data directory : <editme>
OpenIFS GRIB directory: /home/users/auser/ecmwf/eccodes
OpenIFS netCDF directory: /usr/local/lib64
OpenIFS default compiler and build set to: gnu opt
Added OpenIFS FCM and GRIB to user path
Creating command aliases for OpenIFS: omake, omakenew, oenv

Subsequent compilation

With the oifs-config.sh file correctly set, always 'source' this file to set the environment variables before any compilation of OpenIFS:

% source ./oifs-config.sh

FCM

OpenIFS uses software called 'FCM' for the compilation of the code. FCM is contained in the OpenIFS source tarball downloaded from the ftp site.

If you are not using the oifs-config.sh file to set the environment, before trying to compile, make sure the FCM directory is added to your PATH environment variable so the fcm command can be found (if you already have fcm available on your system, we still recommend using the distributed version as this is tested and known to work):

export PATH=$PATH:$HOME/ecmwf/oifs/fcm/bin

If you are not using oifs-config.sh, you will also need to make sure the required OIFS_* environment variables to locate the GRIB and netCDF libraries are set otherwise the compilation will fail.

Build configuration

This is a brief introduction on compiling the OpenIFS model. For a more detailed description of the build system for OpenIFS and the different ways to configure it, see Customizing the OpenIFS build.

Before compiling the model, the build configuration must first be set by several environment variables:

OIFS_COMP - This sets the choice of compiler. The default is 'gnu' which means the gfortran/gcc compilers will be used.  Other choices are 'intel' and 'cce' (Cray compiler).

OIFS_BUILD - This sets the type of build.

These environment variables directly correspond to the names of the FCM configuration files in the make/fcm/ directory in the source. Please see this directory for the choices provided.

Build types: OIFS_BUILD sets the type of build and there will be one FCM configuration file for each type. Build types provided are:

• opt - Recommended optimized compile settings for this compiler. Tested to provide best performance for this platform & compiler.
• noopt - Debugging options. No optimization set for all of the code. Suitable for use with a debugger. OpenMP is disabled.
• nansC - Debugging options with extra checks. Array bound checking enabled and variables initialized to NaN (not-a-number). Use of a variable set to NaN will trigger a floating point exception.  This option is useful for debugging purposes but will run the slowest. It may not work in all cases.

e.g.

export OIFS_COMP=intel
export OIFS_BUILD=opt

means use the Intel ifort compiler and the model will be compiled with full optimization compiler settings. So the FCM build system will expect to find a file:

make/fcm/intel-opt.fcm

Instead setting:

export OIFS_COMP=gnu
export OIFS_BUILD=noopt

means FCM will expect to find a file:

make/fcm/gnu-noopt.fcm

If using GNU on a Mac, please see MacOS X known issues.

Building for other systems

As the environment variables refer to the corresponding filename in the make/fcm  directory it is straightforward to create a build configuration for other environments. Take a copy of an existing file and modify it as necessary. It's also possible to alter compile options by the use of additional environment variables (see below). Which approach you use depends on your personal preference. Be aware however that the optimization compiler flags in the provided configuration files are the recommended ones. It is impossible to test all combinations of compiler flags so the model may become unstable or performance may reduce if higher optimizations are tried.

Setting the location of libraries

The directory paths to the libraries OpenIFS needs for compilation are normally set in the 'oifs-config.sh' file (as described above).  If you choose not to use 'oifs-config.sh', you'll need to set the environment variables that the FCM configuration files expect to be defined.

The model requires a number of libraries (see 5.2 Prerequisites) to compile: the GRIB library (ECMWF's ecCodes), the netCDF (43r3+ only), and the LAPACK & BLAS numerical libraries are required. An MPI library is also needed but this is assumed to be provided by compiler wrappers in the MPI implementation (e.g. the command mpif90 rather than gfortran).

To set the location of the GRIB, netCDF & LAPACK libraries, other environment variables need to be defined as the defaults in the FCM configuration files will most likely not be appropriate.

OIFS_GRIB_DIR - Defines the location of the GRIB library. For more details see '5.3 Install GRIB library for OpenIFS'.

OIFS_NETCDF_DIR - Defines the location of the directory containing the netCDF libraries. This varies from system to system but typically /usr/local/lib. n.b. OpenIFS versions before 43r3 did not use the netCDF library.

OIFS_LAPACK_LIB - Defines the location of the directory containing the liblapack.a and libblas.a files. This varies from system to system but typically /usr/local/lib.

If you do not have LAPACK installed on your system or find it does not work, OpenIFS comes with all the necessary LAPACK code to run. It is disabled by default, see the README.lapack file in the OpenIFS source, or contact openifs-support@ecmwf.int for instructions on how to enable it.

Or if you prefer you can either edit the default value in the appropriate FCM configuration file in make/fcm directory or make a copy of the supplied configuration file (.fcm) and use that.

Compiling OpenIFS

Once the environment variables are set, OpenIFS can be compiled (make sure you have edited your PATH environment variable to add the FCM installation bin directory). Assuming you are in the directory where the OpenIFS source code was unpacked:

% cd make
% fcm make -f oifs.fcm

The command 'fcm make' starts the compilation. The -f option specifies the location of the master configuration file for OpenIFS.

For more information about fcm command options, try: fcm --help .

This generates output similar to:

[init] make                # 2012-12-20 19:59:59Z
[init] make config-parse   # 2012-12-20 19:59:59Z
[info] config-file=...../make/oifs.fcm
[info] config-file= - ...../make/fcm/gnu-opt.fcm
[info] ............
[done] make config-parse   # 0.3s
[init] make dest-init      # 2012-12-20 19:59:59Z
[info] mode=new
[done] make dest-init      # 0.3s
[init] make build          # 2012-12-20 20:00:00Z
[info] sources: total=2194, analysed=2194, elapsed-time=11.7s, total-time=32.2s
[info] target-tree-analysis: elapsed-time=9.8s
[info] compile   targets: modified=2073, unchanged=0, total-time=1095.5s
[info] compile+  targets: modified=628, unchanged=0, total-time=1.6s
[info] ext-iface targets: modified=1247, unchanged=0, total-time=21.9s
[info] install   targets: modified=120, unchanged=0, total-time=0.3s
[info] link      targets: modified=1, unchanged=0, total-time=18.3s
[info] TOTAL     targets: modified=4069, unchanged=0, elapsed-time=316.3s
[done] make build          # 329.1s
[done] make                # 329.8s

Parallel compilation

If you have a multicore processor available, the -j <nthreads> option to 'fcm make' can be used to significantly speed up the compilation of the code.

% fcm make -j 4 -f oifs.fcm	

Tests show that compilation time reduces with a diminishing amount as the number of threads is increased. Set the number of threads to a value less than the available processor cores on your system.

Verbose compilation

The default output of FCM is quite terse. To get more verbose output use the '-v' option.  e.g.

fcm make -v -j 4 -f oifs.fcm

will produce a more detailed report and use 4 processes to analyse and build the model code in parallel.

Successful compilation

After a successful compilation, the OpenIFS executable, master.exe, can be found in the 'make/gnu-opt/oifs/bin' subdirectory (or whichever compiler you choose):

% ls
fcm  gnu-opt
% ls gnu-opt/oifs
bin   include   o
% ls gnu-opt/oifs/bin
master.exe

The 'gnu-opt' subdirectory is generated by FCM using the values of the OIFS_COMP & OIFS_BUILD environment variables. The oifs/include directory contains all the files referenced in the code by a '#include' statement. This includes the Fortran interface block files which are auto-generated by FCM and end in .intfb.h. The oifs/o directory contains all the compiled object files. The OpenIFS build does not place any additional files in the src directory (known as building 'out of source').

If using the intel compiler then the directory would be 'make/intel-opt' and so on.

Compilation problems

If you get the message:

fcm make -f oifs.fcm
/bin/ksh: fcm: not found [No such file or directory]

you need to add the 'bin' directory of the FCM installation to your PATH environment variable. FCM is supplied with OpenIFS and is in the directory 'fcm'.

We recommend editing and using the oifs-config.sh file to store all the correct settings of the OpenIFS environment variables.