Page History

...

1. We are now going to use a simple program that will display versions of different libraries linked to it. Create a file called versions.c using your favourite editor with the following contents:

   Code Block
   language cpp title versions.c collapse true
   #include <stdio.h> #include <hdf5.h> #include <netcdf.h> #include <eccodes.h> int main() { #if defined(__INTEL_LLVM_COMPILER) printf("Compiler: Intel LLVM %d\n", __INTEL_LLVM_COMPILER); #elif defined(__INTEL_COMPILER) printf("Compiler: Intel Classic %d\n", __INTEL_COMPILER); #elif defined(__clang_version__) printf("Compiler: Clang %s\n", __clang_version__); #elif defined(__GNUC__) printf("Compiler: GCC %d.%d.%d\n", __GNUC__, __GNUC_MINOR__, __GNUC_PATCHLEVEL__); #else printf("Compiler information not available\n"); #endif     // HDF5 version unsigned majnum, minnum, relnum; H5get_libversion(&majnum, &minnum, &relnum); printf("HDF5 version: %u.%u.%u\n", majnum, minnum, relnum); // NetCDF version printf("NetCDF version: %s\n", nc_inq_libvers()); // ECCODES version printf("ECCODES version: "); codes_print_api_version(stdout); printf("\n"); return 0; }

   
   

2. Try to naively compile this program with:

   No Format
   $CC -o versions versions.c

   
   

3. The compilation above fails as it does not know where to find the different libraries. We need to add some additional flags so the compiler can find both the include headers and link to the actual libraries. 

   1. Let's use the existing software installed on the system with modules, and benefit from the corresponding environment variables *_DIR which are defined in them to manually construct the include and library flags:

      No Format
      $CC -o versions versions.c -I$HDF5_DIR/include -I$NETCDF4_DIR/include -I/$ECCODESI$ECCODES_DIR/include -L$HDF5_DIR/lib -lhdf5 -L$NETCDF4_DIR/lib -lnetcdf -L$ECCODES_DIR/lib -leccodes

      Load the appropriate modules so that the line above completes successfully and generates the versions executable:

      Expand
      title Solution
      You will need to load the following modules to have those variables defined: No Format module load hdf5 netcdf4 ecmwf-toolbox $CC -o versions versions.c -I$HDF5_DIR/include -I$NETCDF4_DIR/include -I/$ECCODESI$ECCODES_DIR/include -L$HDF5_DIR/lib -lhdf5 -L$NETCDF4_DIR/lib -lnetcdf -L$ECCODES_DIR/lib -leccodes The versions executable should now be in your current directory: No Format ls versions

      
      

   2. Run ./versions. You will get an error such as the one below:

      No Format
      ./versions: error while loading shared libraries: libhdf5.so.200: cannot open shared object file: No such file or directory

      
      

      While you passed the location of the libraries at compile time, the program cannot not find the libraries at runtime. Inspect the executable with ldd to see what libraries are missing

      Expand
      title Solution
      ldd is a utility that prints the shared libraries required by each program or shared library specified on the command line: No Format $ ldd versions linux-vdso.so.1 (0x00007ffffada9000) libhdf5.so.200 => not found libnetcdf.so.19 => not found libeccodes.so => not found libc.so.6 => /lib64/libc.so.6 (0x000014932ff36000) /lib64/ld-linux-x86-64.so.2 (0x00001493302fb000)

      
      

   3. Can you make that program run successfully?

      Expand
      title Solution
      While you passed the location of the libraries at compile time, the program cannot not find the libraries at runtime. There are two solutions: Use the environment variable LD_LIBRARY_PATH- not recommended for long term Use the environment variable LD_LIBRARY_PATH. Check that ldd with the environment variable defined reports all libraries found: No Format LD_LIBRARY_PATH=$HDF5_DIR/lib:$NETCDF4_DIR/lib:$ECCODES_DIR/lib ldd ./versions Rebuild with  rpath  - robust solution  Use the  rpath strategy to engrave the library paths into the actual executable at link time, so it always knows where to find them at runtime. Rebuild your program with: No Format $CC -o versions versions.c -I$HDF5_DIR/include -I$NETCDF4_DIR/include -I/$ECCODES_DIR/include -L$HDF5_DIR/lib -Wl,-rpath,$HDF5_LIB -lhdf5 -L$NETCDF4_DIR/lib -Wl,-rpath,$NETCDF4_DIR/lib -lnetcdf -L$ECCODES_DIR/lib -Wl,-rpath,$ECCODES_DIR/lib -leccodes Check that ldd with the environment variable defined reports all libraries found: No Format unset LD_LIBRARY_PATH ldd ./versions Final version For convenience, all those software modules define the *_INCLUDE and *_LIB variables: No Format module show hdf5 netcdf4 ecmwf-toolbox | grep -e _INCLUDE -e _LIB You can use those in for your compilation directly, with the following simplified compilation line: No Format $CC -o versions versions.c $HDF5_INCLUDE $NETCDF4_INCLUDE $ECCODES_INCLUDE $HDF5_LIB $NETCDF4_LIB $ECCODES_LIB Now you can run your program without any additional settings: No Format $ ./versions Compiler: GCC 8.5.0 HDF5 version: <HDF5 version> NetCDF version: <NetCDF version> of <date> $ ECCODES version: <ecCodes version>

      
      

4. Can you rebuild the program so it uses the "old" versions of all those libraries in modules? Ensure the output of the program matches the versions loaded in modules? Do the same with the latest. 

   Expand
   title Solution
   You need to load the desired versions or the modules: No Format module load hdf5/old netcdf4/old ecmwf-toolbox/old And then rebuild and run the program: No Format $CC -o versions versions.c $HDF5_INCLUDE $NETCDF4_INCLUDE $ECCODES_INCLUDE $HDF5_LIB $NETCDF4_LIB $ECCODES_LIB ./versions The output should match the versions loaded by the modules:  No Format echo $HDF5_VERSION $NETCDF4_VERSION $ECCODES_VERSION Repeat the operation with  No Format module load --latest hdf5 netcdf4 ecmwf-toolbox

   
   

5. To simplify the build process, let's create a simple Makefile for this program. With your favourite editor, create a file called Makefile in the same directory with the following contents:

   Code Block
   language bash title Makefile collapse true
   # # Makefile # # Make sure all the relevant modules are loaded before running make EXEC = hello hello++ hellof versions # TODO: Add the necessary variables into CFLAGS and LDFLAGS definition CFLAGS = LDFLAGS = all: $(EXEC) %: %.c $(CC) -o $@ $^ $(CFLAGS) $(LDFLAGS) %: %.cc $(CXX) -o $@ $^ %: %.f90 $(F90) -o $@ $^ test: $(EXEC) @for exe in $(EXEC); do ./$$exe; done ldd: versions @ldd versions | grep -e netcdf.so -e eccodes.so -e hdf5.so clean: rm -f $(EXEC)

   You can test it works by running:

   No Format
   make clean test ldd

   
   

   Expand
   title Solution
   Edit the Makefile and add the *_INCLUDE and *_LIB variables which are defined by the modules: Code Block language bash title Makefile collapse true # # Makefile # # Make sure all the relevant modules are loaded before running make EXEC = versions CFLAGS = $(HDF5_INCLUDE) $(NETCDF4_INCLUDE) $(ECCODES_INCLUDE) LDFLAGS = $(HDF5_LIB) $(NETCDF4_LIB) $(ECCODES_LIB) all: $(EXEC) %: %.c $(CC) -o $@ $^ $(CFLAGS) $(LDFLAGS) %: %.cc $(CXX) -o $@ $^ %: %.f90 $(F90) -o $@ $^ test: $(EXEC) @for exe in $(EXEC); do ./$$exe; done ldd: versions @ldd versions | grep -e netcdf.so -e eccodes.so -e hdf5.so clean: rm -f $(EXEC) Then run it with: No Format make clean test ldd

   
   

...