Applications et modules d’environnement

Introduction

Before we explain what are the environment modules, you need to understand how are handle the applications on a POSIX system in order to grasp their usefulness on an HPC cluster.

If you do not know the way these applications runs on Linux, you will find the explanation here : Programs and Environment Variables

What is a module ?

An environment module is a file that allows you to describe the running environment of a particular software. It means all the variables and paths necessary for the functionning of the application.

It is also a command (module) that ease the loading and unloading of these applications, compilers and/or frameworks in your session.

Thanks to modules, it is possible to use the right version of a software for the right project.

The cluster IG’s admin give you a batch of modules ready for use and can help you create one for your needs.

Lmod

Historically, the package environment-modules was use on the cluster to handle the modules. They were written in Tcl.

It has been replaced by Lmod and its module files are now written in Lua language. Furthermore, it is compatible with the Tcl modules.

Usage

Listing the available modules

The module avail command allows you to list all the available modules on a server. Those can change depending on the server.

Here are the modules available on the datamaster :

[demo@datamaster ~]$ module avail

---------------------------------------------------------------- /usr/share/lmod/lmod/modulefiles/Core ----------------------------------------------------------------
   lmod/5.8    settarg/5.8

------------------------------------------------------------------------ /storage/modulefiles -------------------------------------------------------------------------
   anaconda3/2019.10

-------------------------------------------------------------------- /shared/software/modules/all ---------------------------------------------------------------------
   Core/M4/1.4.17                                lang/Bison/3.0.4-GCCcore-6.4.0                               numlib/FFTW/3.3.7-gompi-2018a
   Core/M4/1.4.18                         (D)    lang/Bison/3.0.4                                      (D)    numlib/OpenBLAS/0.2.20-GCC-6.4.0-2.28
   Core/help2man/1.47.4                          lang/Java/1.8.0_172                                          numlib/ScaLAPACK/2.0.2-gompi-2018a-OpenBLAS-0.2.20
   Core/zlib/1.2.11                              lang/Python/3.6.4-foss-2018a                                 system/hwloc/1.11.8-GCCcore-6.4.0
   compiler/GCC/6.4.0-2.28                       lang/Tcl/8.6.8-GCCcore-6.4.0                                 toolchain/foss/2018a
   compiler/GCCcore/6.4.0                        lang/flex/2.6.3                                              toolchain/gompi/2018a
   devel/Autoconf/2.69-GCCcore-6.4.0             lang/flex/2.6.4-GCCcore-6.4.0                         (D)    tools/XZ/5.2.3-GCCcore-6.4.0
   devel/Automake/1.15.1-GCCcore-6.4.0           lib/TensorFlow/1.7.1-foss-2018a-Python-3.6.4-CUDA-9.0        tools/binutils/2.28-GCCcore-6.4.0
   devel/Autotools/20170619-GCCcore-6.4.0        lib/TensorFlow/1.7.1-foss-2018a-Python-3.6.4                 tools/binutils/2.28                                (D)
   devel/Bazel/0.11.1-GCCcore-6.4.0              lib/TensorFlow/1.8.0-foss-2018a-Python-3.6.4-CUDA-9.0 (D)    tools/bzip2/1.0.6-GCCcore-6.4.0
   devel/Bazel/0.12.0-GCCcore-6.4.0       (D)    lib/libffi/3.2.1-GCCcore-6.4.0                               tools/gettext/0.19.8.1
   devel/M4/1.4.17                               lib/libreadline/7.0-GCCcore-6.4.0                            tools/help2man/1.47.4-GCCcore-6.4.0
   devel/M4/1.4.18-GCCcore-6.4.0                 lib/libtool/2.4.6-GCCcore-6.4.0                              tools/help2man/1.47.4                              (D)
   devel/M4/1.4.18                        (D)    lib/zlib/1.2.11-GCCcore-6.4.0                                tools/numactl/2.0.11-GCCcore-6.4.0
   devel/SQLite/3.21.0-GCCcore-6.4.0             lib/zlib/1.2.11                                       (D)    tools/wheel/0.30.0-foss-2018a-Python-3.6.4
   devel/ncurses/6.0-GCCcore-6.4.0               math/GMP/6.1.2-GCCcore-6.4.0                                 tools/wheel/0.31.0-foss-2018a-Python-3.6.4         (D)
   devel/ncurses/6.0                      (D)    mpi/OpenMPI/2.1.2-GCC-6.4.0-2.28

  Where:
   (D):  Default Module

Use "module spider" to find all possible modules.
Use "module keyword key1 key2 ..." to search for all possible modules matching any of the "keys".

Enabling a module

The module load command loads an environment module.

Here is an example loading Anaconda :

[demo@datamaster ~]$ conda
conda: command not found

[demo@datamaster ~]$ module load anaconda3

[demo@datamaster ~]$ conda env list
# conda environments:
#
data                     /home_nfs/demo/.conda/envs/data
myownenv                 /home_nfs/demo/.conda/envs/myownenv
base                  *  /storage/modules/anaconda3/2019.10
Pressing the tab key 2 times allows you to complete the module name and show you the available names.

Listing the enabled modules

The module list command print the list of enabled modules :

[demo@datamaster ~]$ module list

Currently Loaded Modules:
  1) anaconda3/2019.10

Disabling a module

The module unload command disable a module :

[demo@datamaster ~]$ module unload anaconda3
[demo@datamaster ~]$ module list
No modules loaded

Disabling the modules

The module purge command disable all active modules :

[demo@datamaster ~]$ module load mpi
mpi/OpenMPI                       mpi/OpenMPI/2.1.2-GCC-6.4.0-2.28
[demo@datamaster ~]$ module load mpi/OpenMPI

[demo@datamaster ~]$ module list

Currently Loaded Modules:
  1) compiler/GCCcore/6.4.0              4) tools/numactl/2.0.11-GCCcore-6.4.0
  2) tools/binutils/2.28-GCCcore-6.4.0   5) system/hwloc/1.11.8-GCCcore-6.4.0
  3) compiler/GCC/6.4.0-2.28             6) mpi/OpenMPI/2.1.2-GCC-6.4.0-2.28

[demo@datamaster ~]$ mpicc
gcc: fatal error: no input files
compilation terminated.

[demo@datamaster ~]$ module purge

[demo@datamaster ~]$ module list
No modules loaded
On the first line, I pressed the tab key 2 times after having written mpi.

Searching for a module

The module spider command allows you to find the modules.

Case sensitive, can return an incomplete list. Always verify with module avail. 
[demo@datamaster ~]$ module spider mpi
-------------------------------------------------------------------------------
  compiler/GCC: compiler/GCC/6.4.0-2.28
-------------------------------------------------------------------------------

-------------------------------------------------------------------------------
  mpi/OpenMPI: mpi/OpenMPI/2.1.2-GCC-6.4.0-2.28
-------------------------------------------------------------------------------

-------------------------------------------------------------------------------
  toolchain/gompi: toolchain/gompi/2018a
-------------------------------------------------------------------------------

Writing your own module

If you add a program in you home directory, you can write a module file to handle its running environment.

For example, we have downloaded a binary archive of the Go language and we extracted its files in the ~/bin/ directory :

[demo@datamaster ~]$ mkdir ~/bin
[demo@datamaster ~]$ wget https://dl.google.com/go/go1.14.3.linux-amd64.tar.gz
[demo@datamaster ~]$ tar -xf go1.14.3.linux-amd64.tar.gz -C ~/bin/

We then create a ~/modulefiles directory in which we will write our module files :

[demo@datamaster ~]$ mkdir -p ~/modulefiles/golang/
[demo@datamaster ~]$ touch modulefiles/golang/1.14.3.lua
[demo@datamaster ~]$ nano modulefiles/golang/1.14.3.lua

Here is the content to add with nano :

Content of ~/modulefiles/golang/1.14.3.lua
help([==[This module sets the PATH variable for golang/1.14.3 ]==])

whatis([==[Description: Go programming language created by Google ]==])
whatis([==[Homepage: https://golang.org ]==])

local goroot = "$HOME/bin/go"

conflict("golang")

setenv("GOROOT", goroot)
prepend_path("PATH", pathJoin(goroot, "bin"))

whatis("Name         : Golang 1.14.3")
whatis("Version      : 1.14.3")
whatis("Category     : Compiler")
whatis("Description  : Go environment ")
whatis("URL          : https://golang.org/ ")

family("golang")

More samples : Modulefile Examples from simple to complex

Once it is saved, we can add the ~/modulefiles directory to the sources of modules with the module use command :

[demo@datamaster ~]$ module use ~/modulefiles
[demo@datamaster ~]$ module load golang
[demo@datamaster ~]$ go version
go version go1.14.3 linux/amd64

If we want to disable the ~/modulefiles folder, it can be done with the module unuse and the absolute path to the directory :

[demo@datamaster ~]$ module unload golang
[demo@datamaster ~]$ module unuse ~/modulefiles

We can automate the loading of the module by adding these lines to the ~/.bashrc file :

if [ -f ~/bin/go/bin/go ] && [-d ~/modulefiles/golang ]; then
    module use ~/modulefiles
    module load golang
fi

If the folders of the language and its module file exists then the golang module will be enabled at connection.

THe home directories are available from every servers.
If these files were on a local directory of the datamaster server (/scratch/ as an example), it would only work for this server and not the others.

Modulefile samples

Written in Lua

help([[
Adds the Julia Programming Language in version 1.3.1 to your environment variables.
]])

local pkgName     = myModuleName()
local fullVersion = myModuleVersion()
local version = "1.3.1"
local base    = "/usr/local/julia-" .. version

whatis("Name: "..pkgName)
whatis("Version: "..fullVersion)
whatis("Description: Julia is a programming language for scientific programming.")
whatis("URL: https://julialang.org")
whatis("Keyword: language, julia, compiler")

setenv("JULIA_BINDIR", pathJoin(base,"bin")) -- /usr/local/julia-1.3.1/bin

prepend_path("PATH", pathJoin(base,"bin"))
prepend_path("LD_LIBRARY_PATH", pathJoin(base,"lib"))
prepend_path("LD_LIBRARY_PATH", pathJoin(base,"lib/julia"))
prepend_path("CPATH", pathJoin(base,"include"))

Written in Tcl

proc ModulesHelp { } {

  puts stderr "\tAdds CUDA Toolkit 9.0.176 to your environment variables,"
}

module-whatis "adds CUDA Toolkit 9.0.176 to your environment variables"

set             root                    /usr/local/cuda-9.0

setenv          CUDA_HOME               $root

prepend-path    PATH                    $root/bin
prepend-path    LD_LIBRARY_PATH         $root/lib
prepend-path    LD_LIBRARY_PATH         $root/lib64:/usr/lib/nvidia-440
prepend-path    PKG_CONFIG_PATH	        $root/pkgconfig