...
Full debugging ('nansC') - this configuration is intended for debugging only. As well as setting the lowest optimization, -O0, it also enables array bound checking and initialisation of variables with special values that will trigger 'not-a-number' exceptions useful for trapping variables used before initialized. This will cause the model to run very slowly and is normally only used for short runs for tracing bugs.
...
Build cfg and variables
The FCM software uses configuration files identified by the suffix '.cfg'. They can be found in the directory oifs/make/fcmcfg
. The master configuration file for OpenIFS is in fcmcfg/oifs.cfg. It sets general configuration options for FCM, lets FCM know where the source code is and reads a single architecture and compiler specific configuration file. No compiler options are contained in the oifs.cfg file - these are always in the architecture specific configuration files.
The naming convention for these architecture specific files is: <architecture>-<compiler>-<build type>.cfg. For example, the file fcmcfg/x86_64-gnu-opt.cfg
would be for Linux 64bit with the GNU compilers and the optimized build. The user can copy and rename the files provided or create their own using an existing one as a template.
How to change build type.
...
FCM would then read the file fcmcfg/oifs.cfg
which would expect to find a file called: darwin-intel-opt.cfg
. This file would need to be created by the user.
...
Compiler options
All the compiler flags and options are contained in the build specific files (e.g. x86_64-intel-opt.cfg). As above, a number of environment variables can be defined to override the default options set in these files.
Ways of working
. As a general rule, any variable with a {?} can be overriden by setting an environment variable.
The following compilation variables in the configuration files can be overriden by environment variables:
User configurable variables | Description |
---|---|
OIFS_GRIB_API_DIR | Location of the grib_api installation directory. This must be changed or overridden to the location of your grib_api install before OpenIFS can be compiled. |
OIFS_GRIB_API_INCLUDE | Location of the 'include' sub-directory in the grib_api installation. This does not normally need changing. |
OIFS_GRIB_API_LIB | Location of the 'lib' sub-directory in the grib_api installation. This does not normally need changing. Note: although grib_api builds both shared and static libraries, OpenIFS always links to the static library in case different versions of grib_api exist on the user system. |
OIFS_LAPACK_LIB | Compiler options to set the directory of the LAPACK and BLAS libraries. This must be changed or overriden as the supplied default is unlikely to be correct on most systems. e.g. OIFS_LAPACK_DIR{?} = -L/opt/apps/lapack/LP64/ -llapack -lblas |
OIFS_EXTRA_LIB | Available in case additional libraries need to be specified. This is most likely set when vendors supply special optimized libraries. One example is IBM's optimized math function library MASS. e.g. OIFS_EXTRA_LIB{?} = -L/usr/lib/mass -lmass -lmassvp6 |
OIFS_FC | Specifies the fortran compiler. The normal default is 'mpif90' but might be 'mpiifort' for the Intel compiler or more generally 'mpifc'. We recommend that mpi compiler wrappers are used whenever possible to simplify the compilation e.g. OIFS_FC = mpif90. |
OIFS_FFLAGS | Fortran compiler flags for free format source files. These flags will vary between the 'opt', 'noopt' and 'nansC' build configurations. e.g. OIFS_FFLAGS = -g -O2 -m64 -march=native -fconvert=big-endian -fopenmp |
OIFS_FFIXED | Extra fortran compiler flags, in addition to OIFS_FFLAGS, for fixed source form Fortran. There are a small number of fortran source files in OpenIFS that are still in fixed format. These options will be added to the options in OIFS_FFLAGS. |
OIFS_FCDEFS | Fortran preprocessor DEFINE flags. This is a list, separated by spaces, of the DEFINE flags required by the code in preprocessor statements like #ifdef. The user is advised not to change these |
OIFS_LFLAGS | Linker options. These are separate flags to be passed to the linker. For example, a common requirement is to generate a loadmap or increase stack memory size. e.g. OIFS_LFLAGS{?} = -bloadmap:map -bmaxstack:0x80000000 |
OIFS_CC | Specifies the C compiler. As for the Fortran compiler this would normally be a compiler wrapper supplied by the MPI system in use. e.g. mpicc for GCC or mpiicc for Intel. |
OIFS_CFLAGS | C compiler flags. As for the Fortran flags, these will vary between the 'opt', 'noopt' and 'nansC' build options. |
OIFS_CCDEFS | C preprocessor flags. As for the OIFS_FCDEFS, the user is advised not to change these. |
Examples
- Set minimum required options.
The only two compilation variables that need to be changed before building OpenIFS are the location of the grib_api installation and the location of the LAPACK and BLAS libraries.
e.g.
export OIFS_GRIB_API_DIR=/home/me/ecmwf/grib_api
export OIFS_LAPACK_LIB="/opt/apps/lapack/current/LP64 -llapack -lblas"
fcm make -f fcmcfg/oifs.cfg
Would override the values of these variables in the FCM config files. In this example, FCM would use the x86_64-gnu-noopt.cfg build as this is the default specified in the oifs.cfg file (in the oifs/make/fcmcfg directory). - Changing compiler and compiler options.
In this example, the build type is changed and the choice of underlying fortran compiler is altered along with some compiler options:
export OIFS_COMP=gnu
export OIFS_BUILD=opt
export OIFS_GRIB_DIR=/opt/local/ecmwf/grib_api_xe6
export OIFS_LAPACK_LIB="-L/opt/apps/scal/lib -llapack -lblas"
export OIFS_FC="mpif90 --fc=crayftn"
export OIFS_FFLAGS="-g -O1 -m64 -convert big-endian"
In this example, the compiler has been changed although FCM will still use the x86_64-gnu-opt.cfg configuration file. This might be because a user wants to temporarily try a different compiler. However, it's recommended that each compiler has its own set of .cfg files rather than alter it this way.
Another example is where an optimized build is required but the optimization level is dropped to -O1 instead of -O2.
Recommendations
As described above, the build environment uses 3 types of build:
- Optimised
- Non-optimized
- Full debugging (nansC)
The compilation of OpenIFS can be changed is several ways:
- Setting the OIFS_ARCH, OIFS_COMP and OIFS_BUILD environment variables.
This changes the compilation to use one of the supplied FCM configuration files: <arch>-<comp>-<build>.cfg. - Set additional environment variables to override the settings in the supplied configuration files.
e.g. changing OIFS_FFLAGS to alter the optimization level. - Creating new .cfg files and set OIFS_ARCH, OIFS_COMP & OIFS_BUILD accordingly. Often best when testing is finished and different compiler options have been chosen.
For temporary changes when testing or debugging, setting environment variables to override the compiler options is perhaps the simplest.
Once testing is finished, or if unsupported compilers are required, then copy an existing .cfg file and create your own. Any names can be used. For instance, if you wanted to have 2 configurations for the Intel compiler, one with the Intel MPI, the other with OpenMPI, then create files: x86_64-intel_impi-opt.cfg and x86_64-intel_ompi-opt.cfg and use the environment variable OIFS_COMP to switch between them. e.g. OIFS_COMP=intel_ompi (it's up to the user whether they want to create .cfg files for each build type; 'opt', 'noopt', 'nansC')Override compiler options for given .cfg files whilst testing and developing. Additional or personalized options once testing is finished can be set in new .cfg files - or - change the ones provided.
HTML |
---|
<script type="text/javascript" src="https://software.ecmwf.int/issues/s/en_UKet2vtj/787/12/1.2.5/_/download/batch/com.atlassian.jira.collector.plugin.jira-issue-collector-plugin:issuecollector/com.atlassian.jira.collector.plugin.jira-issue-collector-plugin:issuecollector.js?collectorId=5fd84ec6"></script> |
...