Overview
From the 2.22.0 version, Magics will be using CMake for its compilation and installation. This is a first step towards an homogenisation of the installation procedures for all ECMWF packages.
Binary versions
Before you install from source code you might want to check that already compiled binary versions are available to you. Magics and third-party dependent software packages might be available as binary packages for you platform in form of RPMs or Debian packages. You might find them by going to the software search services, such as for OpenSuSE and SLES at http://software.opensuse.org/search (select "Search options" - "Include users' home projects"). Ubuntu maintains a Magics version in their system default repository.
CMake installation instructions
The CMake build system is used to build ECMWF software. The build process comprises two stages:
- CMake runs some tests on the system and finds out if required software libraries and headers are available. It uses this information to create native build tools (e.g. Makefiles) for the current platform.
- The actual build can take place, for example by typing '
make'.
Prerequisite
To install any ECMWF software package, CMake needs to be installed on your system. On most systems it will be already installed or this can be done through the standard package manager to install software. For further information to install CMake see
http://www.cmake.org/cmake/help/install.html
Directories
During a build with CMake there are three different directories involved: The source dir, the build dir and the install dir.
| Directory | Use | Example |
|---|---|---|
| Source | Contains the software's source code. This is where a source tarball should be extracted to. | /tmp/src/sw-package |
| Build | Configuration and compiler outputs are generated here, including libraries and executables. | /tmp/build/sw-package |
| Install | Where the software will actually be used from. Installation to this directory is the final stage. | /usr/local |
Of these, the source and build directories can be anywhere on the system. The installation directory is usually left at its default, which is /usr/local. Installing software here ensures that it is automatically available to users. It is possible to specify a different installation directory by adding -DCMAKE_INSTALL_PREFIX=/path/to/install/dir to the CMake command line.
ECMWF software does not support in-source builds. Therefore the build directory cannot be (a subdirectory of) the source directory.
Quick Build Example
Here is an example set of commands to set up and build a software package using default settings. More detail for a customised build is given below.
# unpack the source tarball into a temporary directory mkdir -p /tmp/src cd /tmp/src tar xzvf software-version-Source.tar.gz # configure and build in a separate directory mkdir -p /tmp/build cd /tmp/build cmake /tmp/src/software-version-Source make
On a machine with multiple cores, compilation will be faster by specifying the number of cores to be used simultaneously for the build, for example:
make -j8
If the make command fails, you can get more output by typing:
make VERBOSE=1
The software distribution will include a small set of tests which can help ensure that the build was successful. To start the tests, type:
ctest
As before if you have multiple cores, you can run the tests in parallel by:
ctest -j8
Some projects might not be set up to run tests in parallel. If you experience test failures, run the tests sequentially.
If the tests are successful, you can install the software:
make install
General CMake options
Various options can be passed to the CMake command. The following table gives an overview of some of the general options that can be used. Options are passed to the cmake command by prefixing them with -D, for example -DCMAKE_INSTALL_PREFIX=/path/to/dir.
CMake Option | Description | Default |
|---|---|---|
CMAKE_INSTALL_PREFIX | where to install the software | /usr/local |
CMAKE_BUILD_TYPE | to select the type of compilation:
| RelWithDebInfo (release with debug info) |
CMAKE_CXX_FLAGS | Additional flags to pass to the C++ compiler | |
CMAKE_C_FLAGS | Additional flags to pass to the C compiler | |
CMAKE_Fortran_FLAGS | Additional flags to pass to the Fortran compiler |
The C, C++ and Fortran compilers are chosen by CMake. This can be overwritten by setting the environment variables CC, CXX and F77, before the call to cmake, to set the preferred compiler. Further the variable CMAKE_CXX_FLAGS can be used to set compiler flags for optimisation or debugging. For example, using CMAKE_CXX_FLAGS="-O2 -mtune=native" sets options for better optimisation.
Finding support libraries
If any support libraries are installed in non-default locations, CMake can be instructed where to find them by one of the following methods. First, the option CMAKE_PREFIX_PATH can be set to a colon-separated list of base directories where the libraries are installed, for example -DCMAKE_PREFIX_PATH=/path/where/my/sw/is/installed. CMake will check these directories for any package it requires. This method is therefore useful if many support libraries are installed into the same location.
Troubleshooting
Debugging configure failures
If CMake fails to configure your project, run with debug logging first:
cmake -DECBUILD_LOG_LEVEL=DEBUG [...] /path/to/source
This will output lots of diagnostic information (in blue) on discovery of dependencies and much more.
Requirements
The following table lists the dependency Magics requires to be build from source. Please note, if you install these package from source you also might have to install the respective "-devel" packages.
| Compilers | |||
|---|---|---|---|
| C++ | http://gcc.gnu.org/ | ||
| Fortran | http://gcc.gnu.org/fortran/ | If Fortran or BUFR support needed | |
| Utilities | |||
make | http://www.gnu.org/software/make/ | ||
| cmake | http://www.cmake.org/ | version 2.8.4 minimum | |
| Python Support | |||
| swig | http://www.swig.org/ | to generate the python interface | |
| python | http://www.python.org/ | ||
| python-numpy | http://www.numpy.org/ | ||
| Third party libraries | |||
| proj | http://trac.osgeo.org/proj/ | Proj4 to handle projections | |
| boost | http://www.boost.org/ (Requires 1.5) | used for coastlines | |
| netcdf | http://www.unidata.ucar.edu/software/netcdf/ | if netcdf support needed Please note: You also need to install the legacy C++ interface and HDF5 | |
| pango | http://www.pango.org/ | if png/pdf support needed | |
| expat | http://expat.sourceforge.net/ | for XML parsing | |
| Ecmwf libraries | |||
| grib-api | GRIB-API Home (Requires 1.9) | if grib support needed | |
| odb-api | ODB-API Home | if odb support needed | |
| emoslib | EMOSLIB | if bufr support needed | |
Compilation environment
At ECMWF, OpenSuSE 10.3/11.3 Linux systems (32 and 64bit) were used for testing. Any C++ Compiler which supports features required for the ANSI C++ standard from 1998 (STL, namespaces, templates) should work. At ECMWF we tested GCC’s g++ 4.x successfully. A Fortran compiler is not required for the compilation of Magics, but is needed if you want to use the Fortran interface or compile a dependent Fortran library (Emoslib).
Generating the Makefiles with CMake
After changing into the build Magics directory, the user has to run CMake with his/her own options. The command gives feedback on what requirements are fulfilled and what software is still required. Table below gives an overview of the different options of configure. The default (without any options) will compile a share library only and install it in /usr/local/.
| cmake options | doc | default |
|---|---|---|
| CMAKE_INSTALL_PREFIX | where you want to install your Magics library | /usr/local |
| CMAKE_BUILD_TYPE | to select the type of compilation:
| ? |
| CMAKE_CXX_FLAGS | More flags for the C++ compiler | |
ENABLE_GRIB | enable grib support | on |
| GRIB_API_PATH | where to find grib-api ( if non-standard installation ) | |
| ENABLE_NETCDF | enable netcdf support | on |
| NETCDF4_PATH | where to find netcdf ( if non-standard installation ) | |
| ENABLE_ODB | enable odb support | off |
| ODB_API_PATH | where to find odb ( if non-standard installation ) | |
| ENABLE_BUFR | enable bufr support The bufr support is done through emoslib . | off |
| EMOS_PATH | Where to find emos lib ( if non-standard installation ) | |
| ENABLE_PYTHON | enable python interface | auto |
| ENABLE_FORTRAN | enable fortran interface | on |
| ENABLE_METVIEW | enable metview support(and Qt support) | off |
| ENABLE_CAIRO | enable cairo support | on |
| BOOST_ROOT | where to find boost ( if non-standard installation ) | |
| PROJ4_PATH | where to find proj4 ( if non-standard installation ) |
If auto, CMake will try to enable the feature, but will not fail if it can not.
To make sure that a feature is really enabled, you will have to specify with the option ex: -DENABLE_GRIB=ON. In that case CMake will fail if the grib support can not be enabled.
The C, C++ and Fortran compilers are chosen by CMake. (This can be overwritten by setting the environment variables CC, CXX and FC, on the command line before you call cmake, to the preferred compiler). Further the variable CMAKE_CXX_FLAGS can be used to set compiler flags for optimisation or debugging. For example, it is recommended to use CMAKE_CXX_FLAGS="-O2 -mtune=native" or any other optimisation to compile Magics.
The options to enable/disable output formats allow you to customise your installation. For example, if you have problems on your system with support libraries (see previous section), you might want to try to disable the raster output. Cairo libraries are responsible for most third-party dependencies.
Compiling the code
After the CMake command has run successfully, the user can compile the library by typing make in the build directory.
Testing your build
The Magics code contains a directory called test in which, in separate sub-directories, tests for the various interfaces of Magics are provided. Test programs in Fortran, C, Python and MagML are compiled and run if MAGPLUS_HOME=$PWD make check is invoked from the root directory. (Note that the MAGPLUS_HOME needs to be set!)
The output of the tests should verified before the library is installed. This setup does not check if the user setup is correct, but the code in test can be used to do so. More examples of source code can be found on the Magics web gallery
Installing the library
Once the build and tests have been successfully completed, the command make install copies the library into the correct location on the system. Administrator permission might be required, depending on the installation directory. You might want to run make -n install first, which will show you what will be installed where, without performing any changes to your system.
To free space, the temporary unpacked source directory can be cleaned of the object files with make clean after a successful installation.
Building RPMS
To ease installations over multiple systems or to enable easy re-installation, Magics allows the building of RPMS. By typing the command make rpms a tarball is build and the binary package generated this requires a full compilation. The generated RPMS can be installed with the command sudo rpm -i name.rpm and de-installed by sudo rpm -e name.
Installation FAQ
Why do I get the error message while loading shared libraries: libMagPlus.so.1: cannot open shared object file: No such file or directory when running a Magics executable?
You need to alter your $LD_LIBRARY_PATH variable, as described in section “User setup”. The variable needs to contain the path to your Magics library.
Why do I get a message about a missing / not found libpgc.so when trying to run a Magics executable?
If your EmosLib is compiled with Portland’s Pgf90, it requires this library as shared library at run-time. You need to alter your $LD_LIBRARY_PATH variable.
How can I change the coastline files?
Coastline files are given in Shape file format. The GIS data was taken from http://www.naturalearthdata.com.
Can I add GIS information, such as rivers and borders to my plots?
Yes. Information about political borders and rivers have been added with the new coastline files.
Why have the Printercap file and the frame in my PostScript disappeared?
With version 2.4 of Magics we have changed the set-up of how the PostScript driver was internally organised. The Printercap configuration file was created in the past to accommodate differences between printers. This is nowadays no issue any more. Out of the same reason PostScript output was scaled down to 95% to allow room for Printer alignment problems. From version 2.4 onwards the scaling is, as in all other drivers, set back to 100% of the page size.
Why did version 2.6 of Magics was followed by version 2.8?
With Magics 2.4 we introduced a new numbering scheme where the sub version indicates how stable the version of Magics is. Odd numbers (2.5, 2.7, …) indicate unstable development versions and even numbers (2.4, 2.6, 2.8, …) indicate versions we think are stable and tested enough to be used in your applications.
How can I report bugs or ask for help?
Please write an email to magics@ecmwf.int . Please compress any larger files you might need to attach with gzip.
Quick installation guide
This is an example of a list of commands you could use to install Magics. It is assumed “>” is the shell prompt. The Magics version number may vary.
> tar -xzf Magics++-2.24.0.tar.gz
> cd Magics++-2.24.0
> mkdir build ; cd build
> cmake /path/to/magics-home-directory –DCMAKE_INSTALL_PREFIX=/path/to/where/you/install/Magics++-2.22.0
> make
> setenv MAGPLUS_HOME /path/to/where/you/install/Magics++-2.22.0
> make test << running the test programs
> make install