Page History

Lmod is a Lua-based implementation of the module system, and generally claimed to be backwards compatible with the original. On the newer ECMWF platforms it is replacing the traditional TCL modules.

It is used to configure and manage your session or job environment with all those tools and libraries required for your workload.

Table of contents

Main features

• All the shells in use at the centre are supported.
• Same command and almost identical syntax and options as TCL modules.
• Graceful failure in case of error. TCL modules sometimes fails and gives a zero exit code which cannot be trapped. Some users have implemented their own functions to workaround this issue in the past.
• It does the right thing: automatic swap if different version of module is loaded. With TCL modules you need to know if that module is loaded to decide whether to swap or to load. Many times you end up unloading and loading to make it easier.
• Subcommands like avail or list work on STDOUT so piping the output to other commands such as grep is much more convenient.
• The ml shortcut.
• You can define your own module collections and load them as a whole.
• The ml shortcut.
• Avail Avail mode also shows those modules which are loaded.
• Better support for dependencies and conflicts to other modules.
• Native support to manage different toolchains (different compiler families and versions, and even MPI flavours).

The basics

Behaviour change: Lmod does the right thing

If you have used modules before, the transition should be straightforward. Most commands are the same, although their new behaviour should feel more natural. There are also a few new commands. For a comprehensive list, please run

module -h

module load

Load a module into the current environment of the session or job.

$ module load git
load git 2.20.1 (git_DIR, GIT_VERSION, PATH)

you will see the commands and options are almost identical. You will be able to load and unload modules, as well as checking which ones are loaded or available to load.

Please have a look at the Lmod modules command reference list for details on the different subcommands.

However, there are some changes in the behaviour in respect to the Traditional TCL modules that you may appreciate:

Return codes

If the command fails for any reason (i.e. the module does not exist), it will return a non-zero exit code so you can trap it and deal with it appropriately. There is no need to check wether the module has been loaded or not. You may find that some scripts may fail early where they were not failing before because of this new behaviour

Note that Lmod knows if it is already loaded, and will actually swap unload the old one and load the new one (swapping them)

$ module load git

module swap

Replace one module by another one. This command still works, but load can take care of it as well in the majority of cases.

$ module swap git/new
unload git 2.20.1 (git_DIR, GIT_VERSION, PATH)
load git 2.25.1 (git_DIR, GIT_VERSION, PATH)

The following have been reloaded with a version change:
  1) git/2.20.1 => git/2.25.1

module unload

$ module unload git
unload git 2.25.1 (git_DIR, GIT_VERSION, PATH)

module show

Run this to see what a module would do without loading it. It is very useful to discover what environment variables will be defined so you can use them later on.

; echo return_code=$?
return_code=0
$ module load foo; echo return_code=$?
Lmod has detected the following error:  The following module(s) are unknown: "foo"

Please check the spelling or version number. Also try "module spider ..."
It is also possible your cache file is out-of-date; it may help to try:
  $ module --ignore-cache load "foo"

Also make sure that all modulefiles written in TCL start with the string #%Module

return_code=1

Load and swap

Loading a module when there was another version of it loaded is no longer a problem. Lmod will swap them automatically.

$ module load git
$ module load git/new

The following have been reloaded with a version change:
  1) git/2.20.1 => git/2.25.1

Loaded modules

You can see how many modules and which versions are loaded with "module list"

$ module list

Currently Loaded Modules:
  1) gcc/8.5.0   2) prgenv/gnu   3) git/2.30.1

Automatic version aliases

If you try to load a module with an incomplete version, it will try and find the best (latest) version that matches instead of failing.

$ module avail git

$ module show git
------------------------------------------------------------------------------------------------------ Global Aliases ---------------------------------------------------------------------------------------
   pa /usr/local/apps/modulefiles/lmod/core/git/2.20.1.lua:
-> prgenv/amd    pe -> prgenv/expert    pg -> prgenv/gnu    pi -> prgenv/intel    pp -> prgenv/pgi

--------------------------------------------------------------------------- /usr/local/apps/modulefiles/lmod/core ---------------------------------------------------------------------------------------------------------------
setenv("git_DIR","/usr/local/apps/git/
   git/2.20.1")
setenv("git_VERSION","2.20.1")
setenv("GIT_VERSION","2.20.1")
prepend_path("PATH","/usr/local/apps/git/2.20.1/bin")
LmodMessage("show git 2.20.1 (git_DIR, GIT_VERSION, PATH)")
whatis("Git is a free and open source distributed version control system.")
help([[Git is a free and open source distributed version control system. It is designed to handle everything from small to very large projects with speed and efficiency. For more information on all the available commands run git --help or visit https://git-scm.com/ ]])

module list

See what modules are loaded at any point.

$ module list

Currently Loaded Modules:
  1) git/2.20.1

...

Module list can take an argument to list only those module that match an argument

$ module list

Currently Loaded Modules:
  1) git/2.20.1   2) cmake/3.16.5

$ module list git

Currently Loaded Modules Matching: git
  1) git/2.20.1

 (D)    git/2.25.1 (new)

  Where:
   Aliases:  Aliases exist: foo/1.2.3 (1.2) means that "module load foo/1.2" will load foo/1.2.3
   D:        Default Module

Use "module spider" to find all possible modules and extensions.
Use "module keyword key1 key2 ..." to search for all possible modules matching any of the "keys".

$ module load git/2
$ module load git/2.25

The following have been reloaded with a version change:
  1) git/2.20.1 => git/2.25.1

Conflict management

If two modules cannot be loaded at the same time due to a conflict, the old one will be swapped out and the new one will be loaded automatically.

$ module load proj4
$ module load proj

Lmod is automatically replacing "proj4/5.2.0" with "proj/6.1.1".

Native flavour support

You may feel that some packages or versions are missing when running the classic module avail. Lmod knows about those packages that are flavoured for different environments (compilers or MPI), and will only show as available to load those that have been installed for the currently loaded environment. This guarantees that when loading a module the right flavour of the tool or library will be used, avoiding incompatibilities.

Use module spider to see the whole picture.

The prgenv module

Different compilers families and versions are provided, and with them many different flavours of a given package and versions may be available. Loading the right one matching the current setup avoiding incompatibilities with other packages becomes a complex task.

The prgenv module addresses this problem. There is a prgenv for each compiler family, so it can guarantee a minimal compatibility between different packages.

$ module avail prgenv

---

module avail

List all modules that can be loaded at any point.

You may filter the output by providing a search pattern

$ module avail

-------------------------------------------------------------------------------------- Global Aliases ------------------------------- Global Aliases --------------------------------------------------------
   pa -> prgenv-------------------------------
   pa -> prgenv/amd    pe -> prgenv/expert    pg -> prgenv/gnu    pi -> prgenv/intel    pp -> prgenv/pgi

------------------------------------------------------------------------- /usr/local/apps/modulefiles/lmod/prgenvs --------------------------------------------------------------------------
   prgenv/amd (a)    prgenv/expert (E,e)    prgenv/gnu (D:g)    prgenv/intel (i)    prgenv/pgi

------------------------------------------------------------------------ /usr/local/apps/modulefiles/lmod/compilers -------------------------------------------------------------------------
   aocc/2.1.0    gcc/4.8.5    gcc/8.4.0 (D)    gcc/9.3.0 (new)    intel/18.0.4    intel/19.0.5 (D)    pgi/19.10

--------------------------------------------------------------------------- /usr/local/apps/modulefiles/lmod/core ---------------------------------------------------------------------------
   aec/1.0.3           eigen/3.3.7      git/2.20.1           (L,D)    intel-mkl/18.0.4        lua/5.3.5        nco/4.7.8         proj/6.1.1          valgrind/3.14.0
   antlr/2.7.7         fcm/2019.05.0    git/2.25.1           (new)    intel-mkl/19.0.5 (D)    maven/3.6.3      ninja/1.10.0      proj4/5.2.0
   cmake/3.13.4        ferret/7.5.0     gnuparallel/20190122          jasper/2.0.14           modulemgr/2.0    openjpeg/2.3.0    python3/3.6.8-01
   cmake/3.16.5 (D)    geos/3.7.1       gsl/2.5                       java/11.0.6             nccmp/1.8.2.1    patchelf/0.10     udunits/2.2.26

  Where:
   L:        Module is loaded
   E:        Experimental
   Aliases:  Aliases exist: foo/1.2.3 (1.2) means that "module load foo/1.2" will load foo/1.2.3
   D:        Default Module

Use "module spider" to find all possible modules and extensions.
Use "module keyword key1 key2 ..." to search for all possible modules matching any of the "keys".

$ module avail git

-------------------------------------------------------------------------------------- Global Aliases ---------------------------------------------------------------------------------------
   pa -> prgenv/amd    pe -> prgenv/expert    pg -> prgenv/gnu    pi -> prgenv/intel    pp -> prgenv/pgi

--------------------------------------------------------------------------- /usr/local/apps/modulefiles/lmod/core ---------------------------------------------------------------------------
   git/2.20.1 (L,D)    git/2.25.1 (new)

  Where:
   L:        Module is loaded
   Aliases:  Aliases exist: foo/1.2.3 (1.2) means that "module load foo/1.2" will load foo/1.2.3
   D:        Default Module

Use "module spider" to find all possible modules and extensions.
Use "module keyword key1 key2 ..." to search for all possible modules matching any of the "keys".

You may use the -d option to only list the default version for each module

$ module -d avail

-------------------------------------------------------------------------------------- Global Aliases ---------------------------------------------------------------------------------------
   pa -> prgenv/amd    pe -> prgenv/expert    pg -> prgenv/gnu    pi -> prgenv/intel    pp -> prgenv/pgi

------------------------------------------------------------------------- /usr/local/apps/modulefiles/lmod/prgenvs --------------------------------------------------------------------------
   prgenv/gnu (g)

------------------------------------------------------------------------ /usr/local/apps/modulefiles/lmod/compilers -------------------------------------------------------------------------
   aocc/2.1.0    gcc/8.4.0    intel/19.0.5    pgi/19.10

--------------------------------------------------------------------------- /usr/local/apps/modulefiles/lmod/core ---------------------------------------------------------------------------
   aec/1.0.3           eigen/3.3.7      geos/3.7.1                  gsl/2.5             java/11.0.6    modulemgr/2.0    ninja/1.10.0      proj/6.1.1          udunits/2.2.26
   antlr/2.7.7         fcm/2019.05.0    git/2.20.1           (L)    intel-mkl/19.0.5    lua/5.3.5      nccmp/1.8.2.1    openjpeg/2.3.0    proj4/5.2.0         valgrind/3.14.0
   cmake/3.16.5 (L)    ferret/7.5.0     gnuparallel/20190122        jasper/2.0.14       maven/3.6.3    nco/4.7.8        patchelf/0.10     python3/3.6.8-01

  Where:
   L:        Module is loaded
   Aliases:  Aliases exist: foo/1.2.3 (1.2) means that "module load foo/1.2" will load foo/1.2.3

Use "module spider" to find all possible modules and extensions.
Use "module keyword key1 key2 ..." to search for all possible modules matching any of the "keys".

module whatis

Show a short description of the module

$ module whatis git
git/2.20.1          : Git is a free and open source distributed version control system.

module help

Show a longer description and useful information of the module.

$ module help git

--------------------------------------------------------------------------- Module Specific Help for "git/2.20.1" ---------------------------------------------------------------------------
Git is a free and open source distributed version control system. It is designed to handle everything from small to very large projects with speed and efficiency. For more information on all the available commands run git --help or visit https://git-scm.com/

module spider (new)

List all possible modules, including those that are not visible because they belong to other environments.

$ module spider

-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
The following is a list of the modules and extensions currently available:
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
  aec: aec/1.0.3

  antlr: antlr/2.7.7

  aocc: aocc/2.1.0

  boost: boost/1.61.0:gnu:4.8.5, boost/1.61.0:gnu:8.4, boost/1.61.0:gnu:9.3, boost/1.61.0:intel:19.0, boost/1.61.0

...

You may also enquire for a particular module to see all the versions:

$ module spider fftw

-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
  fftw:
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
     Versions:
        fftw/3.3.8:gnu:4.8
        fftw/3.3.8:gnu:8.4
        fftw/3.3.8:gnu:9.3
        fftw/3.3.8:intel:19.0
        fftw/3.3.8

...

And even more detailed information about a package version in particular:

$ module spider fftw/3.3.8

-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
  fftw: fftw/3.3.8
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

    You will need to load all module(s) on any one of the lines below before the "fftw/3.3.8" module is available to load.

      gcc/4.8.5
      gcc/8.4.0
      gcc/9.3.0
      intel/19.0.5
 
    Help:
      A fast, free C FFT library; includes real-complex, multidimensional, and parallel transforms. For more information visit http://www.fftw.org/ 

module purge

Unload all modules

$ module list

Currently Loaded Modules:
  1) git/2.20.1   2) cmake/3.16.5

$ module purge
unload cmake 3.16.5 (CMAKE_DIR, CMAKE_VERSION, PATH
unload git 2.20.1 (git_DIR, GIT_VERSION, PATH)

ml

Lmod provides a convenience shortcut command called ml that doubles up as module list or any other mode:

...

---------------------------
   prgenv/amd (a)    prgenv/expert (E,e)    prgenv/gnu (D:g)    prgenv/intel (i)    prgenv/pgi

The ECMWF setup: things to remember

Here are a few considerations regarding the setup at ECMWF:

• No modules are loaded by default, beyond the system gcc compiler and associated prgenv. You should load what you need when you need it. If you really want to have some default modules, you may add the corresponding module loads into your ~/.profile or use a collection.
• You may need to load a prgenv, and optionally an MPI implementation, to be able to see a more complete list of modules to load.
• Use module spider if you can't find a module. If it exists, it will tell you how to load it

...

• .