Skip to end of metadata
Go to start of metadata

The Environment Modules package provides the dynamic modification of a user's environment via modulefiles. Modules are useful in managing different versions of applications. They can be loaded and unloaded dynamically and atomically, in an clean fashion. All shells used at the centre are supported, including bash, ksh, and csh.

It is the replacement of the "use" commands that we have used so far in different platforms at ECMWF. In fact, all the main platforms at ECMWF have modules available which can be used, even the ones with the old "use" method in place.

These are the main advantages of using modules instead of use:

  • All the ECMWF supported shells (including bash) work with modules.
  • With modules you can easily see what packages are available to use, and their default versions.
  • You can see what packages have been loaded into your environment
  • You can see what modifications would be done to your environment by a specific module without actually loading them.

For more information see:

http://modules.sourceforge.net/

Contents

Basic usage

These are the most common commands:

List loaded modules

Module list will show you all the modules that are currently loaded into your environment

usxa@m93500:~> module list
Currently Loaded Modulefiles:
  1) version/3.2.10(default)             4) gnu/4.8.1(default)                  7) ecfs/2.0.13rc2(prodn:default:new)  10) ecaccess/4.0.2(default)
  2) verbose/true(default)               5) python/2.7.8-01(default)            8) grib_api/1.12.3(default)           11) emos/394(default)
  3) mode/64(default)                    6) sms/4.4.14(default)                 9) metview/4.5.0(dev:new:default)     12) ecmwf/1.0(default)

List available modules

Module avail will show you all the available modules you could load. You will see as well the default versions of the packages, and also other tags such as new, or dev. in some cases.

usxa@m93500:~> module avail
----------------------------------------- /usr/local/apps/modulefiles/module -----------------------------------------
verbose/false           verbose/true(default)   version/3.2.10(default)
------------------------------------------ /usr/local/apps/modulefiles/init ------------------------------------------
ecmwf/1.0(default) null/1.0
-------------------------------------- /usr/local/apps/modulefiles/environment ---------------------------------------
mode/64(default)
--------------------------------------- /usr/local/apps/modulefiles/compilers ----------------------------------------
clang/3.5.0(default)  gnu/4.8.1(default)    gnu/4.9.1             intel/13.0.1(default) pgi/14.4(default)
------------------------------- /usr/local/apps/modulefiles/tools_and_libraries/ecmwf --------------------------------
Magics/2.22.7(efas:new:default)   eckit/0.6.0(default)              iver/2.1
Magics/2.23.7(daily)              eclib/20140401(default)           iver/2.2
ecaccess/4.0.2(default)           eclib/20140401-r64                iver/2.2.1
ecaccess/4.0.2t                   emos/394(default)                 iver/2.3(new:default)
ecbuild/1.3.0                     emos/394-r64                      metpy/5.2.4(verify:default)
ecbuild/1.3.1                     emos/396                          metpy/5.3.0
ecbuild/1.3.2                     emos/396-r64                      metview/4.4.10
ecbuild/1.4.0(default)            emos/400                          metview/4.4.6
ecflow/4.0.1(default)             emos/400-r64                      metview/4.4.7
ecflow/4.0.2                      fdb/4.6.0                         metview/4.4.8
ecflow/4.0.3                      fdb/4.6.3(default)                metview/4.4.9
ecflow/4.0.4                      grib_api/1.10.0                   metview/4.5.0(dev:new:default)
ecflow/4.0.5                      grib_api/1.11.0(old)              odb/CY40R1.001(default)
ecflow/4.0.6                      grib_api/1.12.0                   odb_api/0.10.1
ecfs/2.0.13rc2(prodn:default:new) grib_api/1.12.1                   odb_api/0.10.2(default)
ecfs/2.2.0(test)                  grib_api/1.12.3(default)          sms/4.4.14(default)
ecfs/2.2.0-rc1                    grib_api/1.13.0(new)
---------------------------- /usr/local/apps/modulefiles/tools_and_libraries/data_formats ----------------------------
cdo/1.6.3               hdf5/1.8.12             nccmp/1.4.0(default)    netcdf4/4.3.1.1
cdo/1.6.4(default)      hdf5/1.8.13(default)    ncview/2.1.2(default)   netcdf4/4.3.2(default)
hdf/4.2.10(default)     ncbrowse/1.6.7(default) netcdf/3.6.3(default)
----------------------------- /usr/local/apps/modulefiles/tools_and_libraries/utilities ------------------------------
R/3.0.3                        grads/2.0.2                    pv-wave/10.0a(default)
R/3.1.1(default)               gsl/1.16(default)              pyngl/1.4.0(default)
aec/0.3(default)               idl/8.2(default)               python/2.7.6-01
boost/1.53.0                   latex/2014(default)            python/2.7.8-01(default)
boost/1.55.0(default)          matlab/R2010a(default)         qgis/2.6.0(default)
boost/1.57.0(new)              matlab/R2013a(new)             rttov/11.2(default)
fcm/2014.09.0(default)         nag/23(default)                szip/2.1(default)
ferret/6.9(default)            nag/24(new)                    totalview/tv8.12.0-0
fftw/3.3.4(default)            ncl/6.2.0(default)             totalview/tv8.14.0-16(default)
gdal/1.10.1                    nco/4.4.4                      udunits/2.2.12
gdal/1.11.0                    nco/4.4.6(default)             udunits/2.2.17(default)
gdal/1.11.1(default)           panoply/4.0(default)           wgrib/current(default)
geos/3.4.2(default)            pcraster/4.0.1(default)        xconv/1.91(default)
grads/2.0.1(default)           proj4/4.8.0(default)

Load and unload a module

To add some package to your environment, just module load (or add) it. You can remove it from your environment with module unload (or remove). If you don't specify any version, the default will be loaded.

usxa@m93500:~> module load Magics
load Magics 2.22.7 Library (MAGPLUS_HOME, MAG_LOGGING_FILE, MAGPLUS_DEV, MAGPLUS_INFO,  MAGPLUS_VERSION, MAGPLUSLIB_STATIC, MAGPLUSLIB_SHARED, MAGPLUSLIB_STATIC_DOUBLE, MAGPLUSLIB_SHARED_DOUBLE )
usxa@m93500:~> module unload Magics
remove Magics 2.22.7 Library (MAGPLUS_HOME, MAG_LOGGING_FILE, MAGPLUS_DEV, MAGPLUS_INFO,  MAGPLUS_VERSION, MAGPLUSLIB_STATIC, MAGPLUSLIB_SHARED, MAGPLUSLIB_STATIC_DOUBLE, MAGPLUSLIB_SHARED_DOUBLE )

You can load specific versions/tags of a module by specifying both the name and the version:

usxa@m93500:~> module load metview/new
load metview 4.5.0 (PATH)
usxa@m93500:~> module unload metview
remove metview 4.5.0 (PATH)
usxa@m93500:~> module load metview/4.5.0
load metview 4.5.0 (PATH)

Switch versions of a module

You will notice that when a module is loaded, you cannot load a different version of the same module:

usxa@m93500:~> module load grib_api/1.13.0
grib_api/1.13.0(34):ERROR:150: Module 'grib_api/1.13.0' conflicts with the currently loaded module(s) 'grib_api/1.12.3'
grib_api/1.13.0(34):ERROR:102: Tcl command execution failed: conflict $current_package

If that happens, you can use switch (or swap)

usxa@m93500:~> module swap grib_api grib_api/1.13.0
switch1 grib_api 1.12.3 Library (GRIB_API_MODE, GRIB_API_VERSION, GRIB_API_INCLUDE, GRIB_API_LIB, GRIB_API_INCLUDE_DIR, GRIB_API_LIB_DIR, EC_FFLAGS, EC_CFLAGS, EC_CXXFLAGS, EC_FLDFLAGS, EC_CLDFLAGS, EC_CXXLDFLAGS)
switch2 grib_api 1.13.0 Library (GRIB_API_MODE, GRIB_API_VERSION, GRIB_API_INCLUDE, GRIB_API_LIB, GRIB_API_INCLUDE_DIR, GRIB_API_LIB_DIR, EC_FFLAGS, EC_CFLAGS, EC_CXXFLAGS, EC_FLDFLAGS, EC_CLDFLAGS, EC_CXXLDFLAGS)

More advanced usage

Display what a module would do

If you want to see what a module would do without having to load it, you can just use display or show:

usxa@m93500:~> module show grib_api
-------------------------------------------------------------------
/usr/local/apps/modulefiles/tools_and_libraries/ecmwf/grib_api/1.12.3:
conflict         grib_api 
setenv           grib_api_DIR /usr/local/apps/grib_api/1.12.3/GNU/4.8.1 
setenv           GRIB_API_DIR /usr/local/apps/grib_api/1.12.3/GNU/4.8.1 
prepend-path     EC_FFLAGS_INTERNAL -I/usr/local/apps/grib_api/1.12.3/GNU/4.8.1/include 
prepend-path     EC_CFLAGS_INTERNAL -I/usr/local/apps/grib_api/1.12.3/GNU/4.8.1/include  
prepend-path     EC_CXXFLAGS_INTERNAL -I/usr/local/apps/grib_api/1.12.3/GNU/4.8.1/include  
prepend-path     EC_CXXLDFLAGS_INTERNAL -L/usr/local/apps/grib_api/1.12.3/GNU/4.8.1/lib -lgrib_api -ljasper  
prepend-path     EC_CLDFLAGS_INTERNAL -L/usr/local/apps/grib_api/1.12.3/GNU/4.8.1/lib -lgrib_api -ljasper  
prepend-path     EC_FLDFLAGS_INTERNAL -L/usr/local/apps/grib_api/1.12.3/GNU/4.8.1/lib -lgrib_api_f90 -lgrib_api -ljasper 
append-path      EC_CXXLDFLAGS_INTERNAL -lm 
append-path      EC_CLDFLAGS_INTERNAL -lm 
setenv           GRIB_API_PATH /usr/local/apps/grib_api/1.12.3/GNU/4.8.1 
setenv           GRIB_API_DIR /usr/local/apps/grib_api 
setenv           GRIB_API_MODE 64 
setenv           GRIB_API_VERSION 1.12.3 
setenv           GRIB_API_INCLUDE -I/usr/local/apps/grib_api/1.12.3/GNU/4.8.1/include  
setenv           GRIB_API_LIB -L/usr/local/apps/grib_api/1.12.3/GNU/4.8.1/lib -lgrib_api_f90 -lgrib_api -ljasper 
setenv           GRIB_API_INCLUDE_DIR /usr/local/apps/grib_api/1.12.3/GNU/4.8.1/include 
setenv           GRIB_API_LIB_DIR /usr/local/apps/grib_api/1.12.3/GNU/4.8.1/lib 
prepend-path     PATH /usr/local/apps/grib_api/1.12.3/GNU/4.8.1/bin 
module-whatis    Set environment variables to enable the usage of the grib_api 1.12.3 data formats library. 
-------------------------------------------------------------------

Reset the module session to the default

To get the environment back to the default, use purge (which will unload all the modules), and then load the ecmwf module

usxa@m93500:~> module purge && module load ecmwf
remove emos 394 (PATH, EMOSLIB, EC_FFLAGS, EC_FLDFLAGS,EC_CLDFLAGS, EC_CXXLDFLAGS)
remove ecaccess 4.0.2 (PATH,MANPATH)
remove grib_api 1.13.0 Library (GRIB_API_MODE, GRIB_API_VERSION, GRIB_API_INCLUDE, GRIB_API_LIB, GRIB_API_INCLUDE_DIR, GRIB_API_LIB_DIR, EC_FFLAGS, EC_CFLAGS, EC_CXXFLAGS, EC_FLDFLAGS, EC_CLDFLAGS, EC_CXXLDFLAGS)
remove ecfs 2.0.13rc2 Library (PATH,MANPATH,ECFS_SYS_PATH,ECUMASK,ECDIR,EC_TMPDIR)
remove sms 4.4.14 (PATH,XCDPHOME, LD_LIBRARY_PATH)
remove python 2.7.8-01 (PATH)
remove gnu 4.8.1 (EC_COMPILER_FAMILY, EC_COMPILER_VERSION, CC, CXX, FC, F77, F90)
remove mode 64 bit (OBJECT_MODE)
remove verbose true (usage of the verbose mode)
load verbose true (usage of the verbose mode)
load mode 64 bit (OBJECT_MODE)
load gnu 4.8.1 (EC_COMPILER_FAMILY, EC_COMPILER_VERSION, CC, CXX, FC, F77, F90)
load python 2.7.8-01 (PATH)
load sms 4.4.14 (PATH,XCDPHOME, LD_LIBRARY_PATH)
load ecfs 2.0.13rc2 Library (PATH,MANPATH,ECFS_SYS_PATH,ECUMASK,ECDIR,EC_TMPDIR)
load grib_api 1.12.3 Library (GRIB_API_MODE, GRIB_API_VERSION, GRIB_API_INCLUDE, GRIB_API_LIB, GRIB_API_INCLUDE_DIR, GRIB_API_LIB_DIR, EC_FFLAGS, EC_CFLAGS, EC_CXXFLAGS, EC_FLDFLAGS, EC_CLDFLAGS, EC_CXXLDFLAGS)
load metview 4.5.0 (PATH)
load ecaccess 4.0.2 (PATH,MANPATH)
load emos 394 (PATH, EMOSLIB, EC_FFLAGS, EC_FLDFLAGS,EC_CLDFLAGS, EC_CXXLDFLAGS)

Change the compiler

If you want to change the compiler for your session, you can do it with switch/swap or unloading the current one and loading the new one. That will trigger the reload of all the loaded modules that require a different environment for the new compiler:

usxa@m93500:~> module swap gnu pgi
switch1 gnu 4.8.1 (EC_COMPILER_FAMILY, EC_COMPILER_VERSION, CC, CXX, FC, F77, F90)
switch1 grib_api 1.12.3 Library (GRIB_API_MODE, GRIB_API_VERSION, GRIB_API_INCLUDE, GRIB_API_LIB, GRIB_API_INCLUDE_DIR, GRIB_API_LIB_DIR, EC_FFLAGS, EC_CFLAGS, EC_CXXFLAGS, EC_FLDFLAGS, EC_CLDFLAGS, EC_CXXLDFLAGS)
switch2 pgi 14.4 (EC_COMPILER_FAMILY, EC_COMPILER_VERSION, CC, CXX, FC, F77, F90)
switch2 grib_api 1.12.3 Library (GRIB_API_MODE, GRIB_API_VERSION, GRIB_API_INCLUDE, GRIB_API_LIB, GRIB_API_INCLUDE_DIR, GRIB_API_LIB_DIR, EC_FFLAGS, EC_CFLAGS, EC_CXXFLAGS, EC_FLDFLAGS, EC_CLDFLAGS, EC_CXXLDFLAGS)

On the CRAY supercomputer, the above command won't work. The PrgEnvs should be swapped instead:

usxa@cca-login2:~> module swap PrgEnv-cray PrgEnv-gnu

The utilites prgenvswitchto and prgenvinfo are also available on all platforms with support for multiple compilers:

usxa@m93500:~> prgenvinfo 
GNU/4.8.1
usxa@m93500:~> prgenvswitchto pgi
switch1 gnu 4.8.1 (EC_COMPILER_FAMILY, EC_COMPILER_VERSION, CC, CXX, FC, F77, F90)
switch1 grib_api 1.12.3 Library (GRIB_API_MODE, GRIB_API_VERSION, GRIB_API_INCLUDE, GRIB_API_LIB, GRIB_API_INCLUDE_DIR, GRIB_API_LIB_DIR, EC_FFLAGS, EC_CFLAGS, EC_CXXFLAGS, EC_FLDFLAGS, EC_CLDFLAGS, EC_CXXLDFLAGS)
switch2 pgi 14.4 (EC_COMPILER_FAMILY, EC_COMPILER_VERSION, CC, CXX, FC, F77, F90)
switch2 grib_api 1.12.3 Library (GRIB_API_MODE, GRIB_API_VERSION, GRIB_API_INCLUDE, GRIB_API_LIB, GRIB_API_INCLUDE_DIR, GRIB_API_LIB_DIR, EC_FFLAGS, EC_CFLAGS, EC_CXXFLAGS, EC_FLDFLAGS, EC_CLDFLAGS, EC_CXXLDFLAGS)
usxa@m93500:~> prgenvinfo 
PGI/14.4

Configure a set of modules to be loaded by default

If you want to load a set of modules by default, you can add the corresponding module commands to your .user_kshrc, .user_bashrc, or .cshrc depending what is your login shell. Here is an example:

module load Magics
module swap eccodes/new 

the modules you define there will be loaded in all plaftorms sharing your same HOME directory. If the modules you try to load are not available on certain platforms, you would get an error at login time. That is why you might want to load different sets of modules depending on the platform you are in, either using the OS version or the hostname. Here is an example where those modules will only be loaded on the Leap42 desktops:

if [ `os_version` == "leap42" ]; then
    module load Magics
    module swap eccodes/new
fi

Or for csh users:

if ( `os_version` == "leap42" ) then
    module load Magics
    module swap eccodes/new 
endif

 

 

2 Comments

  1. If anyone has problems switching between compilers: The command:

    module swap PrgEnv-cray PrgEnv-gnu

    does not work reliably. Libraries such as grib-api remain loaded for the old compiler. We had to manually unload all the libraries which appear after PrgEnv-cray in `module list` via:

    module unload grib-api/..., ..., ..., PrgEnv-cray

    and then load the gnu compiler and the other modules in reverse order:

    module load PrgEnv-gnu, ..., ..., ... grib-api/...

     

    1. Alternatively, an easier way to achieve the same is to use the helper command prgenvswitchto as described above. This command will automatically reload the libraries that require reloading after switching the Programming Environment.