Versions Compared

Old Version 103

changes.mady.by.user Unknown User (nagc)

Saved on

compared with

New Version 104

changes.mady.by.user Unknown User (nagc)

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

 

Section


Column

Introduction

The ECMWF grib_api software library provides a set of functions/subroutines and command line tools for encoding and decoding WMO FM-92 GRIB edition 1 and edition 2 messages. The OpenIFS models use GRIB for its input and output files.

The grib_api software library must be built and installed before compiling any OpenIFS model that needs it.

If you have a version of grib_api on your system already that would most likely work, though check the minimum grib-api version required for each model.  It is recommended that you build and install grib-api as described below to ensure the correct configuration is used. Also, we recommend compiling your own to ensure the same compiler and compiler version is used for the model, as otherwise, error messages about conflicting module versions when compiling can arise.

For more information about grib_api, to download and install it please visit the grib_api website.

Before you start

Decide where you will put the grib_api package.

For a single user, grib_api might be installed in your home directory. For a multi-user environment, say a cluster or High Performance Computing Facility (HPCF), installing the grib_api package centrally once only is probably best and pointing your build configuration to this location.

In the walk-through example that follows, a single-user download and install is assumed.


Column
width50px

 


Column
width280px


Panel
bgColorwhite
titleBGColorlightlightgrey
titleOn this page

Table of Contents
maxLevel2
printablefalse



Download and unpack

Create a directory called $HOME/ecmwf in which grib-api will be downloaded and compiled.

If you haven't already obtained the grib_api package, get it from the grib-api web site. We recommend downloading the latest version. Some of the instructions for options below may differ with the very latest versions.

Unpacking grib_api

To unpack the software:

Code Block
languagebash
titleCreate directory and unpack source
mkdir -p ecmwf/src
cd ecmwf/src
tar zxf grib_api-1.14.0-Source.tar.gz

...

Code Block
gunzip grib_api_1.14.0.tar.gz
tar xf grib_api-1.14.0.tar

Configuring using CMake

Grib_api uses the CMake command for configuration and compilation. CMake is available for modern Linux systems but may have to be installed.

...

CMAKE_C_COMPILER=gcc
CMAKE_Fortran_COMPILER=gfortran

These lines set the choice of compiler. CMake is often able to determine the available compilers for itself.
However, in cases where multiple compilers are available, or where compiler wrappers are used (as on HPC systems), it's preferable to set these explicitly.
These lines can often be omitted. Note the CMake variables are preferred rather than set environment variables CC and FC.
CMAKE_INSTALL_PREFIX="../grib_api"

This specifies the location where the 'make install' command will place the grib-api installation.

ENABLE_NETCDF=ONEnabling netcdf ensures the grib_to_netcdf command can be used.
ENABLE_JPG=OFF
ENABLE_PNG=OFF		As OpenIFS models do not deal with any image based GRIB data these options can be disabled, unless you have other grib data containing images that you need to work with.
ENABLE_PYTHON=ONSome of the utilities that come with the OpenIFS model make use of the python interface to grib_api, e.g. the tools to create and manipulate the model data.
For this option to work, ensure that a python installation is available. This option is not needed for the single column nor offline surface models currently.
ENABLE_FORTRAN=ONThis enables the Fortran interface to grib_api and should always be on.
ENABLE_GRIB_OMP_THREADS=ONNot essential but recommended. This ensures grib_api supports OpenMP threads. Note this is not the same as the option ENABLE_GRIB_THREADS which is for Posix threading and not used by the OpenIFS model. Although OpenIFS models do not currently use multiple threaded calls to grib-api, future versions may do so this option is recommended.
BUILD_SHARED_LIBS=OFFgrib_api can be built with both 'shared' and 'static' libraries for linking. Normally static libraries are sufficient. Note that some IBM users may experience problems if shared libraries are built.

Available options

To get additional help on the available options (similar to the 'configure --help' command) run the command:

...

Info

This command will cause cmake to run its system discovery if this is the first time the command has been used. On some systems, this may take time to complete.

Configuring using autotools

Grib_api compilation can be made using the GNU configure tool. This will be familiar to many who have previously installed software.

...

CC=gcc FC=gfortranThese set the choice of C and Fortran compilers. They can be omitted as the configure script will usually find them. However, if you have multiple compilers available and want to use a particular compiler, use these environment variables.
--prefix        
This specifies where you want the grib_api files to be installed. In example here, the source code is in $HOME/ecmwf/grib_api_1.14.0-Source and the compiled grib_api libraries will go in directory $HOME/ecmwf/grib_api. You are free to choose any installation directory but it's recommended to keep the installed binaries & libraries in a separate directory to the source code, in case different compiler versions are needed. On a shared high performance computer facility, the install path would most likely be somewhere central. If nothing is specified for --prefix, by default grib_api will install in /usr/local.
--disable-jpegOpenIFS models do not need to support JPEG and PNG for data compression. This removes the need to link against the Jasper library (libjasper.a).
--enable-pthreadThis option ensures that grib_api is thread-safe and is a recommended option.
--enable-pythonThis option is not essential but desirable. It enables the python interface and is required as some utilities make use of the grib-api python interface. If the make fails because of a missing numpy header file we recommend installing numpy. Although you could add the --disable-numpy option, if this option does not work, some of the tools provided with OpenIFS models may not work.

Available options

--help option to configure can be used to see what other options are possible, also see the grib_api website for more detailed documentation.

--enable-vector option should be used on vector based computers (e.g. NEC). Grib_api will then use a more efficient packing/unpacking method suitable for vector hardware. n.b. there is a bug in the 1.9.18 release that if the --disable-vector option is used, this has the reverse effect of enabling the vector code. See grib_api issue GRIB-269 for more details..

Shared and static libraries

grib_api will build both shared and static libraries by default. Shared libraries may need to be built if the python interface is used, depending on your installation.

...

IBM users may find that the grib-api fortran tests will fail unless --disable-shared is used.

Preferred compiler options

By default, cmake and configure will use the preferred compiler and set appropriate compiler options accordingly (for cmake, this depends on the build type).

...

Panel
bgColorlightlightgrey
borderStyledashed

Compiler options if using configure

Expand
titleclick here to expand...


Code Block
titlegcc/gfortran compilers
export CC=gcc
export FC=gfortran
export CCFLAGS="-g -O2"
export FCFLAGS="-g -O2"
./configure ...


Code Block
titleIntel compiler:
export CC=icc
export FC=ifort
export CFLAGS="-g -O1 -fp-model precise"
export FCFLAGS="-g -O1 -fp-model precise"
./configure ...


Code Block
languagebash
titleCray compiler (cce)
export CC=cc
export FC=ftn
export CFLAGS="-O1 -G2 -hflex_mp=conservative -hadd_paren -hfp1"
export FCFLAGS="-O1 -G2 -hflex_mp=conservative -hadd_paren -hfp1"
./configure ...


Code Block
titleIBM compiler:
export CC=xlc_r
export FC=xlf90_r
export CFLAGS="-g -O2 -qstrict -qarch=auto -qtune=auto"
export FCFLAGS="-g -O2 -qstrict -qarch=auto -qtune=auto"
./configure ..



Compiling, checks and installing

After configuration the next steps are to compile grib-api and install it. To compile grib_api do:

...

After this stage, next steps are to download and install FCM and then download the OpenIFS code and accompanying files.

Possible problems

Info
titleFor help contact...

 OpenIFS support: openifs-support@ecmwf.int.

Cray

Possible issues with the Cray systems (using cce or gnu compilers) are related to building shared libraries and linking with supporting netcdf and python libraries.

...

LIBRARY_PATH is only used at compile/link time. It may also be necessary to add this directory to the LD_LIBRARY_PATH environment variable which is searched for libraries at runtime.

Intel compiler

We currently recommend not using a higher optimization level than -O1 with the Intel compiler. Failures have been seen with grib_api and the Intel compiler when compiled with -O2.

Some versions of the Intel compiler (v14.0.x) can produce an error "unknown option -soname" when using 'configure. If this occurs try using "--disable-shared" or try the cmake command instead.

IBM compiler

For the IBM xlc compiler we recommend disabling the creation of 'shared libraries' which is known to cause problems with some versions of grib_api.

In this example, note the use of the '_r' form of the IBM compiler to ensure grib_api is compiled 'thread-safe'.

Building on High Performance Computer Systems

Some HPC batch systems have a different hardware architecture for their login (or frontend node) to the batch node, but the frontend compilation system is targeted at the batch nodes. This is known as cross-compilation. If this is the case you may see failures in the 'make check' stage because the checks, although compiled for the backend batch nodes, are being run on the frontend nodes and therefore may not work correctly. If this is the case on your system, we recommend using a batch job to do the 'configure; make; make check; make install' steps.

...

If you have any questions installing grib_api in this type of environment, please contact: openifs-support@ecmwf.int for assistance.

Verify the installation

To verify that the installation was successful, first ensure that the grib_api bin directory is added to your PATH environment variable and then run the grib_info command e.g.

...