Software

Shabyt HPC cluster provides a large and diverse set of software, including various compilers, programming languages, libraries, and applications. Users can also install software of their choosing themselves in their home directories.

The following guides provide instructions for finding and loading software, building and installing software, as well as using popular applications.

Software Installation

Software installation on the Shabyt system follows specific criteria to ensure compatibility and effective utilization of resources. Users can request the installation of new software if it meets the following conditions:

• Availability and Licensing: The software must be freely available or covered by a site license held by NU.
• Compatibility: It should be compatible with the existing operating system environment on Shabyt to ensure seamless integration and functionality.
• Resource Utilization: The software should be able to effectively utilize the resources available on Shabyt, optimizing performance and efficiency.

For guidance or support regarding the installation of new software packages, users should contact the Shabyt system administrators at hpcadmin@nu.edu.kz.

Additionally, software are installed in accordance with priorities.

• Priority 1: Software that can be installed using the EasyBuild application is given first priority. A list of supported EasyBuild software can be found here.
• Priority 2: Applications which can't be installed through EasyBuild, but essential for multiple User Groups are prioritized next.
• Priority 3: Application which can't be installed through EasyBuild, but essential for individual users.

It's important to know that this isn't a complete list of all the software in Shabyt system.

Software list

• Environment Module System
• Anaconda
• GCC
• CUDA
• MATLAB
• Apptainer

Environment Modules

In linux environment variables are values that can change and impact how programs behave on a computer system. They are name-value pairs that all processes can access within a particular user environment or shell session. These variables provide a flexible and convenient method for managing system-wide settings, configuring applications, and customizing system behavior.

Shabyt uses Environment modules (also know as Lmod) to dynamically set up environment variables for different software. Module commands set, change, or delete environment variables that are needed for a particular software. The ‘module load‘ command will set PATH, LD_LIBRARY_PATH and other environment variables such that user may choose a desired version of applications or libraries more easily. More details can be found here.

Environment module commands

Command	Description
module avail	List of available software
module keyword [word]	Search for available modules matching the keyword
module spider [word]	Show the details of any modules matching the keyword
module whatis [module]	Show the short description about module
module load [package1] [package2]	Load the environment for the default version of the modulefile
module load [package]/[version]	Load the environment for the specified version of module
module unload [package1] [package2]	Unload previously loaded packages
module swap [moduleA] [moduleB]	Unload modulefile A and load modulefile B
module list	List any currently loaded module(s)
module purge	Unload all currently loaded modules

Using module avail

To see what modules you can load into your environment, enter the command module avail.

[hpcadmin@ln01 ~]$ module avail

----------------------------------------------------------------------------- /shared/opt/easybuild/modules/all -----------------------------------------------------------------------------
   AMD-uProf/3.5.671                                       PLUMED/2.7.3-foss-2021b                                          help2man/1.49.3-GCCcore-12.3.0
   AOCC/4.0.0-GCCcore-12.2.0                               PMIx/3.1.5-GCCcore-9.3.0                                         help2man/1.49.3-GCCcore-13.2.0          (D)
   ATK/2.38.0-GCCcore-11.3.0                               PMIx/3.2.3-GCCcore-10.3.0                                        hwloc/2.2.0-GCCcore-9.3.0
   Anaconda3/2022.05                                       PMIx/4.1.0-GCCcore-11.2.0                                        hwloc/2.2.0-GCCcore-10.2.0
   Autoconf/2.69-GCCcore-9.3.0                             PMIx/4.1.2-GCCcore-11.3.0                                        hwloc/2.4.1-GCCcore-10.3.0
   Autoconf/2.69-GCCcore-10.2.0                            PMIx/4.2.2-GCCcore-12.2.0                                        hwloc/2.5.0-GCCcore-11.2.0
   Autoconf/2.71-GCCcore-10.3.0                            PMIx/4.2.6-GCCcore-13.2.0                               (L,D)    hwloc/2.7.1-GCCcore-11.3.0
   Autoconf/2.71-GCCcore-11.2.0                            PROJ/8.0.1-GCCcore-10.3.0                                        hwloc/2.8.0-GCCcore-12.2.0
   Autoconf/2.71-GCCcore-11.3.0                            PROJ/9.0.0-GCCcore-11.3.0                               (D)      hwloc/2.9.2-GCCcore-13.2.0              (L,D)
   Autoconf/2.71-GCCcore-12.2.0                            Pango/1.48.5-GCCcore-10.3.0                                      hypothesis/6.13.1-GCCcore-10.3.0
   Autoconf/2.71-GCCcore-12.3.0                            Pango/1.48.8-GCCcore-11.2.0                                      hypothesis/6.14.6-GCCcore-11.2.0
   Autoconf/2.71-GCCcore-13.2.0                   (D)      Pango/1.50.7-GCCcore-11.3.0                             (D)      hypothesis/6.46.7-GCCcore-11.3.0        (D)

To unload all your loaded modules, enter the command module purge. Then module listwill return No modules loaded.

Using module spider

If you know the name of a software package, you can use the module spidercommand to find out if it is available and how to load it.

For example, to search for Python modules:

[hpcadmin@ln01 ~]$ module spider python

-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
  Python:
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    Description:
      Python is a programming language that lets you work more quickly and integrate your systems more effectively.

     Versions:
        Python/2.7.18-GCCcore-11.3.0-bare
        Python/3.8.6-GCCcore-10.2.0
        Python/3.9.5-GCCcore-10.3.0-bare
        Python/3.9.5-GCCcore-10.3.0
        Python/3.9.6-GCCcore-11.2.0-bare
        Python/3.9.6-GCCcore-11.2.0
        Python/3.10.4-GCCcore-11.3.0-bare
        Python/3.10.4-GCCcore-11.3.0
        Python/3.11.3-GCCcore-12.3.0
        Python/3.11.5-GCCcore-13.2.0

-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
  For detailed information about a specific "Python" package (including how to load the modules) use the module's full name.
  Note that names that have a trailing (E) are extensions provided by other modules.
  For example:

     $ module spider Python/3.11.5-GCCcore-13.2.0
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

This shows that there are multiple versions of Python available. For more specific information, add the version to your command as given in the example:

[hpcadmin@ln01 ~]$ module spider Python/3.11.5-GCCcore-13.2.0

-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
  Python: Python/3.11.5-GCCcore-13.2.0
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    Description:
      Python is a programming language that lets you work more quickly and integrate your systems more effectively.


    This module can be loaded directly: module load Python/3.11.5-GCCcore-13.2.0

    Help:

      Description
      ===========
      Python is a programming language that lets you work more quickly and integrate your systems
       more effectively.


      More information
      ================
       - Homepage: https://python.org/


      Included extensions
      ===================
      flit_core-3.9.0, packaging-23.2, pip-23.2.1, setuptools-68.2.2, setuptools-
      scm-8.0.4, tomli-2.0.1, typing_extensions-4.8.0, wheel-0.41.2

Using module keyword

If you do not know the exact name of a software package, you can use the module keywordcommand instead to search for modules.

[hpcadmin@ln01 ~]$ module keyword python
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

The following modules match your search criteria: "python"
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

  Anaconda3: Anaconda3/2022.05
    Built to complement the rich, open source Python community, the Anaconda platform provides an enterprise-ready data analytics platform that empowers companies to adopt a modern
    open data science analytics architecture.


  Python: Python/2.7.18-GCCcore-11.3.0-bare, Python/3.8.6-GCCcore-10.2.0, Python/3.9.5-GCCcore-10.3.0-bare, Python/3.9.5-GCCcore-10.3.0, Python/3.9.6-GCCcore-11.2.0-bare, ...
    Python is a programming language that lets you work more quickly and integrate your systems more effectively.

Loading and unloading software

Typically, loading modules is as simple as entering module load <software_name>. The <software_name>must be visible when you run module avail. Lmod will set your environment such that the software specified in <software_name>will be placed in your PATH, and then you can run the commands associated with <software_name>. If there are multiple versions of <software_name>, you can specify a version. For example:

[hpcadmin@ln01 ~]$ module load GCC
[hpcadmin@ln01 ~]$ module list

Currently Loaded Modules:
  1) GCCcore/13.2.0   2) zlib/1.2.13-GCCcore-13.2.0   3) binutils/2.40-GCCcore-13.2.0   4) GCC/13.2.0


[hpcadmin@ln01 ~]$ module load GCC/11.3.0

The following have been reloaded with a version change:
  1) GCC/13.2.0 => GCC/11.3.0             3) binutils/2.40-GCCcore-13.2.0 => binutils/2.38-GCCcore-11.3.0
  2) GCCcore/13.2.0 => GCCcore/11.3.0     4) zlib/1.2.13-GCCcore-13.2.0 => zlib/1.2.12-GCCcore-11.3.0

[hpcadmin@ln01 ~]$ module list

Currently Loaded Modules:
  1) GCCcore/11.3.0   2) zlib/1.2.12-GCCcore-11.3.0   3) binutils/2.38-GCCcore-11.3.0   4) GCC/11.3.0

If no version is specified, then the default version will be loaded. The default version is indicated with a (D)next to it after running module avail.:

To unload a specific module, enter:

[hpcadmin@ln01 ~]$ module unload GCC/11.3.0