Lmod is a Lua-based implementation of the module system, and generally claimed to be backwards compatible with the original. On the newer ECMWF platforms it is replacing the traditional TCL modules.
| Table of Contents |
|---|
Main features
- All the shells in use at the centre are supported.
- Same command and almost identical syntax and options as TCL modules.
- Graceful failure in case of error. TCL modules sometimes fails and gives a zero exit code which cannot be trapped. Some users have implemented their own functions to workaround this issue in the past.
- It does the right thing: automatic swap if different version of module is loaded. With TCL modules you need to know if that module is loaded to decide whether to swap or to load. Many times you end up unloading and loading to make it easier.
- Subcommands like avail or list work on STDOUT so piping the output to other commands such as grep is much more convenient.
- The ml shortcut.
- Avail mode also shows those modules which are loaded.
- Better support for dependencies and conflicts to other modules.
- Native support to manage different toolchains (different compiler families and versions, and even MPI flavours).
The tool basics
...
...
No modules are loaded by default
If you have used modules before, the transition should be straightforward. Most commands are the same, although their new behaviour should feel more natural. There are also a few new commands. For a comprehensive list, please run
| No Format |
|---|
module -h |
module load
Load a module into the current environment of the session or job.
| No Format |
|---|
$ module load git
load git 2.20.1 (git_DIR, GIT_VERSION, PATH) |
...
Note that Lmod knows if it is already loaded, and will actually swap unload the old one and load the new one (swapping them)
| No Format |
|---|
$ module load git/new
unload git 2.20.1 (git_DIR, GIT_VERSION, PATH)
load git 2.25.1 (git_DIR, GIT_VERSION, PATH)
The following have been reloaded with a version change:
1) git/2.20.1 => git/2.25.1 |
module swap
Replace one module by another one. This command still works, but load can take care of it as well in the majority of cases.
| No Format |
|---|
$ module swap git/new
unload git 2.20.1 (git_DIR, GIT_VERSION, PATH)
load git 2.25.1 (git_DIR, GIT_VERSION, PATH)
The following have been reloaded with a version change:
1) git/2.20.1 => git/2.25.1 |
module unload
| No Format |
|---|
$ module unload git
unload git 2.25.1 (git_DIR, GIT_VERSION, PATH) |
module show
Run this to see what a module would do without loading it. It is very useful to discover what environment variables will be defined so you can use them later on.
| No Format |
|---|
$ module show git
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
/usr/local/apps/modulefiles/lmod/core/git/2.20.1.lua:
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
setenv("git_DIR","/usr/local/apps/git/2.20.1")
setenv("git_VERSION","2.20.1")
setenv("GIT_VERSION","2.20.1")
prepend_path("PATH","/usr/local/apps/git/2.20.1/bin")
LmodMessage("show git 2.20.1 (git_DIR, GIT_VERSION, PATH)")
whatis("Git is a free and open source distributed version control system.")
help([[Git is a free and open source distributed version control system. It is designed to handle everything from small to very large projects with speed and efficiency. For more information on all the available commands run git --help or visit https://git-scm.com/ ]]) |
module list
See what modules are loaded at any point.
| No Format |
|---|
$ module list
Currently Loaded Modules:
1) git/2.20.1 |
| Tip |
|---|
Note that Lmod will also mark what modules are loaded when running module avail (with an L next to them) |
...
Module list can take an argument to list only those module that match an argument
| No Format |
|---|
$ module list
Currently Loaded Modules:
1) git/2.20.1 2) cmake/3.16.5
$ module list git
Currently Loaded Modules Matching: git
1) git/2.20.1 |
module avail
List all modules that can be loaded at any point.
| Warning |
|---|
module avail will not show all the possible modules, but only those that can be loaded in the current environment. To see all possible modules and their versions, use module spider |
| No Format |
|---|
$ module avail
-------------------------------------------------------------------------------------- Global Aliases ---------------------------------------------------------------------------------------
pa -> prgenv/amd pe -> prgenv/expert pg -> prgenv/gnu pi -> prgenv/intel pp -> prgenv/pgi
------------------------------------------------------------------------- /usr/local/apps/modulefiles/lmod/prgenvs --------------------------------------------------------------------------
prgenv/amd (a) prgenv/expert (E,e) prgenv/gnu (D:g) prgenv/intel (i) prgenv/pgi
------------------------------------------------------------------------ /usr/local/apps/modulefiles/lmod/compilers -------------------------------------------------------------------------
aocc/2.1.0 gcc/4.8.5 gcc/8.4.0 (D) gcc/9.3.0 (new) intel/18.0.4 intel/19.0.5 (D) pgi/19.10
--------------------------------------------------------------------------- /usr/local/apps/modulefiles/lmod/core ---------------------------------------------------------------------------
aec/1.0.3 eigen/3.3.7 git/2.20.1 (L,D) intel-mkl/18.0.4 lua/5.3.5 nco/4.7.8 proj/6.1.1 valgrind/3.14.0
antlr/2.7.7 fcm/2019.05.0 git/2.25.1 (new) intel-mkl/19.0.5 (D) maven/3.6.3 ninja/1.10.0 proj4/5.2.0
cmake/3.13.4 ferret/7.5.0 gnuparallel/20190122 jasper/2.0.14 modulemgr/2.0 openjpeg/2.3.0 python3/3.6.8-01
cmake/3.16.5 (D) geos/3.7.1 gsl/2.5 java/11.0.6 nccmp/1.8.2.1 patchelf/0.10 udunits/2.2.26
Where:
L: Module is loaded
E: Experimental
Aliases: Aliases exist: foo/1.2.3 (1.2) means that "module load foo/1.2" will load foo/1.2.3
D: Default Module
Use "module spider" to find all possible modules and extensions.
Use "module keyword key1 key2 ..." to search for all possible modules matching any of the "keys". |
...
You may filter the output by providing a search pattern
| No Format |
|---|
$ module avail git
-------------------------------------------------------------------------------------- Global Aliases ---------------------------------------------------------------------------------------
pa -> prgenv/amd pe -> prgenv/expert pg -> prgenv/gnu pi -> prgenv/intel pp -> prgenv/pgi
--------------------------------------------------------------------------- /usr/local/apps/modulefiles/lmod/core ---------------------------------------------------------------------------
git/2.20.1 (L,D) git/2.25.1 (new)
Where:
L: Module is loaded
Aliases: Aliases exist: foo/1.2.3 (1.2) means that "module load foo/1.2" will load foo/1.2.3
D: Default Module
Use "module spider" to find all possible modules and extensions.
Use "module keyword key1 key2 ..." to search for all possible modules matching any of the "keys".
|
you will see the commands and options are almost identical. You will be able to load and unload modules, as well as checking which ones are loaded or available to load.
Please have a look at the Lmod modules command reference list for details on the different subcommands.
Behaviour change: Lmod does the right thing
Return codes
If the command fails for any reason (i.e. the module does not exist), it will return a non-zero exit code so you can trap it and deal with it appropriately. There is no need to check wether the module has been loaded or not. You may find that some scripts may fail early where they were not failing before because of this new behaviour
Load and swap
Loading a module when there was another version of it loaded is no longer a problem. Lmod will swap them automatically.
Conflict management
If two modules cannot be loaded at the same time due to a conflict, the old one will be swapped out and the new one will be loaded automatically.
Native flavour support
You may feel that some packages or versions are missing when running the classic module avail. Lmod knows about those packages that are flavoured for different environments (compilers or MPI), and will only show as available to load those that have been installed for the currently loaded environment. Use module spider to see the whole picture
The ECMWF setup
| Note |
|---|
Note that there are no modules are loaded by default |
...
You may use the -d option to only list the default version for each module
| No Format |
|---|
$ module -d avail
-------------------------------------------------------------------------------------- Global Aliases ---------------------------------------------------------------------------------------
pa -> prgenv/amd pe -> prgenv/expert pg -> prgenv/gnu pi -> prgenv/intel pp -> prgenv/pgi
------------------------------------------------------------------------- /usr/local/apps/modulefiles/lmod/prgenvs --------------------------------------------------------------------------
prgenv/gnu (g)
------------------------------------------------------------------------ /usr/local/apps/modulefiles/lmod/compilers -------------------------------------------------------------------------
aocc/2.1.0 gcc/8.4.0 intel/19.0.5 pgi/19.10
--------------------------------------------------------------------------- /usr/local/apps/modulefiles/lmod/core ---------------------------------------------------------------------------
aec/1.0.3 eigen/3.3.7 geos/3.7.1 gsl/2.5 java/11.0.6 modulemgr/2.0 ninja/1.10.0 proj/6.1.1 udunits/2.2.26
antlr/2.7.7 fcm/2019.05.0 git/2.20.1 (L) intel-mkl/19.0.5 lua/5.3.5 nccmp/1.8.2.1 openjpeg/2.3.0 proj4/5.2.0 valgrind/3.14.0
cmake/3.16.5 (L) ferret/7.5.0 gnuparallel/20190122 jasper/2.0.14 maven/3.6.3 nco/4.7.8 patchelf/0.10 python3/3.6.8-01
Where:
L: Module is loaded
Aliases: Aliases exist: foo/1.2.3 (1.2) means that "module load foo/1.2" will load foo/1.2.3
Use "module spider" to find all possible modules and extensions.
Use "module keyword key1 key2 ..." to search for all possible modules matching any of the "keys". |
module whatis
Show a short description of the module
| No Format |
|---|
$ module whatis git
git/2.20.1 : Git is a free and open source distributed version control system. |
module help
Show a longer description and useful information of the module.
| No Format |
|---|
$ module help git
--------------------------------------------------------------------------- Module Specific Help for "git/2.20.1" ---------------------------------------------------------------------------
Git is a free and open source distributed version control system. It is designed to handle everything from small to very large projects with speed and efficiency. For more information on all the available commands run git --help or visit https://git-scm.com/ |
module spider (new)
List all possible modules, including those that are not visible because they belong to other environments.
| No Format |
|---|
$ module spider
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
The following is a list of the modules and extensions currently available:
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
aec: aec/1.0.3
antlr: antlr/2.7.7
aocc: aocc/2.1.0
boost: boost/1.61.0:gnu:4.8.5, boost/1.61.0:gnu:8.4, boost/1.61.0:gnu:9.3, boost/1.61.0:intel:19.0, boost/1.61.0
... |
You may also enquire for a particular module to see all the versions:
| No Format |
|---|
$ module spider fftw
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
fftw:
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Versions:
fftw/3.3.8:gnu:4.8
fftw/3.3.8:gnu:8.4
fftw/3.3.8:gnu:9.3
fftw/3.3.8:intel:19.0
fftw/3.3.8
... |
And even more detailed information about a package version in particular:
| No Format |
|---|
$ module spider fftw/3.3.8
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
fftw: fftw/3.3.8
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
You will need to load all module(s) on any one of the lines below before the "fftw/3.3.8" module is available to load.
gcc/4.8.5
gcc/8.4.0
gcc/9.3.0
intel/19.0.5
Help:
A fast, free C FFT library; includes real-complex, multidimensional, and parallel transforms. For more information visit http://www.fftw.org/
|
module purge
Unload all modules
| No Format |
|---|
$ module list
Currently Loaded Modules:
1) git/2.20.1 2) cmake/3.16.5
$ module purge
unload cmake 3.16.5 (CMAKE_DIR, CMAKE_VERSION, PATH
unload git 2.20.1 (git_DIR, GIT_VERSION, PATH)
|
ml
Lmod provides a convenience shortcut command called ml that doubles up as module list or any other mode:
| No Format |
|---|
| $ ml Currently Loaded Modules: 1) git/2.20.1 $ ml git/new unload git 2.20.1 (git_DIR, GIT_VERSION, PATH) load git 2.25.1 (git_DIR, GIT_VERSION, PATH) The following have been reloaded with a version change: 1) git/2.20.1 => git/2.25.1 $ ml Currently Loaded Modules: 1) git/2.25.1 $ ml whatis git git/2.25.1 : Git is a free and open source distributed version control system. |