Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Section
Column

Introduction

The ECMWF grib_api software library provides a set of functions/subroutines and command line tools for encoding and decoding WMO FM-92 GRIB edition 1 and edition 2 messages. The OpenIFS model uses GRIB for its input and output files.

The grib_api software library must be build and installed before building OpenIFS. Although there may be a version of grib_api on your system already that would most likely work, it is recommended that you build and install grib-api yourself as described below to ensure the correct configuration is used for OpenIFS. Also, to ensure the same compiler and compiler version is used. Otherwise you may see , as otherwise, error messages about conflicting module versions when compiling OpenIFS can arise.

This page gives additional information about grib-api for OpenIFS. For more information about grib_api, to download and install it please visit the grib_api website.

Column
width50px

 

Column
width280px
Panel
bgColorwhite
titleBGColorlightlightgrey
titleOn this page

Table of Contents
maxLevel2
stylesquare
printablefalse

...

For a single user, grib_api might be installed in your home directory. For a multi-user environment, say a cluster or High Performance Computing Facility (HPCF), we suggest installing the grib_api package centrally once only is probably best and pointing your build configuration to this location.

In the walk-through example that follows, we assume a single-user download and install is assumed.

Download and unpack

In this example, we assume you have created create a directory called $HOME/ecmwf in which you will download and compile grib_apigrib-api will be downloaded and compiled.

If you haven't already obtained the grib_api package, get it from the grib-api web site.

...

Code Block
languagebash
titleSteps to configure grib-api with CMake assuming gnu compilers
mkdir bld
mkdir grib-api
cd bld
export CC=gcc
export FC=gfortran
cmake ../grib_api-1.14.0-Source   \
      -DCMAKE_BUILD_TYPE=Bit          \
      -DCMAKE_INSTALL_PREFIX="../grib-api"   \
      -DENABLE_NETCDF=ON              \
      -DENABLE_JPG=OFF                \
      -DENABLE_PNG=OFF                \
      -DENABLE_PYTHON=ON              \
      -DENABLE_FORTRAN=ON             \
      -DENABLE_GRIB_THREADS=ON

...

export CC=gcc
export FC=gfortran

These lines set the choice of compiler. CMake is often able to determine the available compilers for itself.
However, in cases where multiple compilers are available, or where compiler wrappers are used (as on HPC systems), it's preferable to set these explicitly by use of environment variables.
These lines can often be omitted.
CMAKECMAKE_BUILD_TYPE=Bit"Bit" here means 'bit-reproducible'. It instructs CMake to use conservative compiler options that are known to ensure bit-reproducibility in separate runs of OpenIFS.
CMAKE_INSTALL_PREFIX="../grib_api"

This specifies the location where the 'make install' command will place the grib-api installation.

ENABLE_NETCDF=ONEnabling netcdf ensures the grib_to_netcdf command can be used.
ENABLE_JPG=OFF
ENABLE_PNG=OFF
As OpenIFS does not deal with any image based GRIB data these options can be disabled, unless you have other grib data containing images that you need to work with.
ENABLE_PYTHON=ONSome of the utilities that come with OpenIFS make use of the python interface to grib_api, e.g. the tools to create and manipulate the model data.
For this option to work, ensure that a python installation is available.
ENABLE_FORTRAN=ONThis enables the Fortran interface to grib_api and should always be on.
ENABLE_GRIB_THREADS=ONThis ensures grib_api is compiled to be 'thread-safe'. Although OpenIFS does not currently use multiple threaded calls to grib-api, future versions may do so this option is recommended.

 

Available options

To get additional help on the available options (similar to the 'configure --help' command) run the command:

Code Block
cmake -LH grib_api-1.14.0-Source
Info

This command will cause cmake to run its system discovery if this is the first time the command has been used. On some systems, this may take time to complete.

 

Configuring using configure

Grib_api compilation can also be made using the GNU configure tool to ease building and compilation. It is tested on different platforms and compilers and should not cause any problems. If however, you do have a problem installing grib_api, please email: openifs-support@ecmwf.int.

Other build types

CMake supports several standard 'build types'. These are:

 

ReleaseRelease mode - the binary package does not include debug symbols and some of the source code is ignored; this mode is the default for users.
DebugDebug mode - the binary package includes debug symbols in the source code are not ignored; this mode is the default for developers and typically uses low or no optimisation.
RelWithDebInfoRelease mode, but the binary package includes debug symbols this mode is for developers and users and is often the default build type.
BitBit-reproducibility mode - This is non-standard and will not be found on other cmake enabled packages. It is used at ECMWF to set compiler options that have been tested to provide bit-reproducibility.

 

Available options

To get additional help on the available options (similar to the 'configure --help' command) run the command:

Code Block
cmake -LH grib_api-1.14.0-Source
Info

This command will cause cmake to run its system discovery if this is the first time the command has been used. On some systems, this may take time to complete.

Configuring using configure

Grib_api compilation can also be made using the GNU configure tool.

The configure The configure command is contained in the grib_api directory you have just unpacked. We will use the command with a . A few options are needed to suit OpenIFS and assume we'll use the gcc & gfortran compilers are assumed in the example below. If you use different compilers, see below.

Code Block
titlegcc/gfortran compilers
cd grib_api_1.914.180-Source
./configure --prefix=$HOME/ecmwf/grib_api --enable-pthread --enable-python --disable-jpeg

...

A quick description of what these options are for:

--prefix        
This specifies where you want the grib_api files to be installed. In our example here, the source code is in $HOME/ecmwf/grib_api_1.14.0-Source and we will put the compiled grib_api libraries will go in our example directory $HOME/ecmwf/grib_api_gcc. You are free to choose any installation directory but it's recommended to keep the installed binaries & libraries in a separate directory to the source code, in case different compiler versions are needed. On a shared high performance computer facility, the install path would most likely be somewhere central. If nothing is specified for --prefix, by default grib_api will install in /usr/local.
--disable-jpegOpenIFS does not need to support JPEG and PNG for data compression. This removes the need to link OpenIFS against the Jasper library (libjasper.a).
--enable-pthreadThis option ensures that grib_api is thread-safe and is a recommended option.
--enable-pythonThis enables the python interface and is required as some OpenIFS utilities make use of the grib-api python interface. If the make fails because of a missing numpy header file you should add the --disable-numpy option.

...

grib_api will build both shared and static libraries by default. Shared libraries may need to be built if the python interface is used, depending on your installation.

...

By default, cmake and configure will use the preferred compiler and set appropriate compiler options accordingly (for cmake, this depends on the build type).

(by searching your PATH environment variable). This may not be what you need if you have multiple compilers available . As in and want to direct cmake/configure to the most appropriate one to use. In the above example, we enforce the use of the GNU C and Fortran compilers (as this is the default compiler for OpenIFS). If you intend to use other compilers with OpenIFS you will need to set these appropriately.

Recommended options are conservative to ensure stability and bit-reproducibility. The cmake and configure commands usually chooses choose a sensible set of optimized compiler flags for the choice of supported compiler .for the 'Bit' build type.

Although it is not normally necessary to specify them, the The following examples show the recommended compiler options and how to set them for the cmake and configure commands.

Portland compiler:IBM

For the IBM xlc compiler we recommend disabling the creation of 'shared libraries' which is known to cause problems with some versions of grib_api.

In this example, note the use of the '_r' form of the IBM compiler to ensure grib_api is compiled 'thread-safe'.
Section
Column
Panel
borderColorwhite
bgColorwhite
titleBGColorwhite
borderStyledotted
titleIf using 'configure' command
 
Code Block
titlegcc/gfortran compilers
export CC=gcc
export FC=gfortran
export CCFLAGS="-g -O2"
export FCFLAGS="-g -O2"
Code Block
titleIntel compiler:
export CC=icc
export FC=ifort
export CFLAGS="-g -O1 -fp-model precise"
export FCFLAGS="-g -O1 -fp-model precise"
Note

We currently recommend not using a higher optimization level than -O1 with the Intel compiler. Failures have been seen with grib_api and the Intel compiler when compiled with -O2.

Code Block
languagebash
titleCray compiler (cce)
export CC=cc
export FC=ftn
export CFLAGS="-O1 -G2 -hflex_mp=conservative -hadd_paren -hfp1"
export FCFLAGS="-O1 -G2
-hflex_mp=conservative -hadd_paren -hfp1"
Code Block
 -hflex_mp=conservative -hadd_paren -hfp1"
Code Block
titleIBM compiler:
export CC=xlc_r
export FC=xlf90_r
export CFLAGS="-g -O2 -qstrict -qarch=auto -qtune=auto"
export FCFLAGS="-g -O2 -qstrict -qarch=auto -qtune=auto"
Note

For the IBM xlc compiler we recommend disabling the creation of 'shared libraries' which is known to cause problems with some versions of grib_api.

In this example, note the use of the '_r' form of the IBM compiler to ensure grib_api is compiled 'thread-safe'.

Column
Panel
borderColorwhite
bgColorwhite
titleBGColorwhite
borderStyledotted
titleIf using 'cmake' command
Code Block
titlegcc/gfortran compilers
title
export CC=
pgcc
gcc
export FC=
pgf90 export CFLAGS
gfortran
cmake .... -DCMAKE_C_FLAGS="-g -O2"
export FCFLAGS
 -DCMAKE_Fortran_FLAGS="-g -O2"
Code Block
title
Intel compiler:
export CC
=xlc_r export FC=xlf90_r export CFLAGS="-g -O2 -qstrict -qarch=auto -qtune=auto" export FCFLAGS
=icc
export FC=ifort
export CFLAGS="-g -
O2 -qstrict -qarch=auto -qtune=auto"
Note
O1 -fp-model precise"
export FCFLAGS="-g -O1 -fp-model precise"

 

 

Compiling, checks and installing

...

Excerpt Include
Credits
Credits
nopaneltrue