Software 2

This page gives an overview of the software available in NU HPC facilities and explains how to use it.

Software Installation

Software installation on the Shabyt system follows specific criteria to ensure compatibility and effective utilization of resources. Users can request the installation of new software if it meets the following conditions:

• Availability and Licensing: The software must be freely available or covered by a site license held by NU.
• Compatibility: It should be compatible with the existing operating system environment on Shabyt to ensure seamless integration and functionality.
• Resource Utilization: The software should be able to effectively utilize the resources available on Shabyt, optimizing performance and efficiency.

For guidance or support regarding the installation of new software packages, users should contact the Shabyt system administrators at hpcadmin@nu.edu.kz.

Additionally, software are installed in accordance with priorities.

• Priority 1: Software that can be installed using the EasyBuild application is given first priority. A list of supported EasyBuild software can be found here.
• Priority 2: Applications which can't be installed through EasyBuild, but essential for multiple User Groups are prioritized next.
• Priority 3: Application which can't be installed through EasyBuild, but essential for individual users.

It's important to know that this isn't a complete list of all the software in Shabyt system.

Environment Modules

In linux environment variables are values that can change and impact how programs behave on a computer system. They are name-value pairs that all processes can access within a particular user environment or shell session. These variables provide a flexible and convenient method for managing system-wide settings, configuring applications, and customizing system behavior.

Shabyt uses Environment modules (also know as LMOD) to dynamically set up environment variables for different software. Module commands set, change, or delete environment variables that are needed for a particular software. The ‘module load‘ command will set PATH, LD_LIBRARY_PATH and other environment variables such that user may choose a desired version of applications or libraries more easily. More details can be found here.

Environment module commands

Command	Description
module avail	List of available software
module keyword [word]	Search for available modules matching the keyword
module spider [word]	Show the details of any modules matching the keyword
module whatis [module]	Show the short description about module
module load [package1] [package2]	Load the environment for the default version of the modulefile
module load [package]/[version]	Load the environment for the specified version of module
module unload [package1] [package2]	Unload previously loaded packages
module swap [moduleA] [moduleB]	Unload modulefile A and load modulefile B
module list	List any currently loaded module(s)
module purge	Unload all currently loaded modules

Anaconda

Description: Anaconda, also known as "conda," is a tool for managing Python packages. It helps you create virtual environments for different Python and package versions. You can use Anaconda to install, remove, and update packages within your project environments. For instance you can create virtual environment for game development which requires Pygame with version of Python and you can create environment for machine learning which requires Pytorch with new version of Python.

Usage: module load Anaconda3/2022.05

Working with Anaconda environments

Below is a list of main commands you should use in order to start working with Anaconda.

1. To Check available environments, please type: conda env list
2. View a list of packages in an environment
   • If the environment is not activated, please type: conda list -n virtualenv
   • If the environment is activated, then type: conda list
3. Create Conda environment
   • Create an environment: conda create -n virtualenv
   • Create an environment with a specific Python version: conda create -n virtualenv python=3.12
   • Create an environment to target directory: conda create -p /shared/home/{username}/.conda/envs/virtualenv
4. Activate an environment: source activate virtualenv
5. Deactivate an environment: conda deactivate
6. Remove an environment

conda remove -n virtualenv --all or conda env remove -n virtualenv

Working with packages

Install packages into virtualenv environment

• If the environment is not activated, please type: conda --name virtualenv install PACKAGENAME
• If the environment is activated, please type: conda install PACKAGENAME
• If you want to install multiple packages at once: conda install pkg1 pkg2 pkg3
• If you need to install package with specific version: conda install numpy=1.15.2

External links

Documentation

User Guide

Video

Conda Cheat Sheet

CUDA

CUDA (Compute Unified Device Architecture) is a parallel computing platform and application programming interface model created by NVIDIA. It allows to use NVIDIA graphics processing units (GPUs) for general purpose processing.

Usage: module load cuda/11.4.1

To check GPUs on GPU nodes: nvidia-smi --list-gpus

GCC

The GNU Compiler Collection, commonly known as GCC, is a set of compilers and development tools available for Linux, Windows, various BSDs, and a wide assortment of other operating systems. It includes support primarily for C and C++ and includes Objective-C, Ada, Go, Fortran, and D. The Free Software Foundation (FSF) wrote GCC and released it as completely free (as in libre) software.

GCC is a toolchain that compiles code, links it with any library dependencies, converts that code to assembly, and then prepares executable files. It follows the standard UNIX design philosophy of using simple tools that perform individual tasks well. The GCC development suite utilizes these discrete tools to compile software.

When you run GCC on a source code file, it first uses a preprocessor to include header files and discard comments. Next, it tokenizes the code, expands macros, detects any compile-time issues, then prepares it for compilation. It is then sent to the compiler, which creates syntax trees of the program’s objects and control flow and uses those to generate assembly code. The assembler then converts this code into the binary executable format of the system. Finally, the linker includes references to any external libraries as needed. The finished product is then executable on the target system.

GCC examples

Compiling a program with GCC can be a straightforward matter

gcc hello.c -o hello

Running this command processes the hello.c file and generates a binary called “hello”.

Additional parameters can be passed.

gcc hello.c -O3 -o hello

In this example, the optimization parameter is set to 3, leading to more optimized code generation.

More complex compilations are managed by Makefiles and are invoked with the “make” command.

External link

Official Page

Apptainer

Singularity is an open-source application for creating and running software containers, designed primarily for high-performance computing on shared Linux-based computing clusters like CARC systems. Singularity containers provide a custom user space and enable portable, reproducible, stable, and secure software environments on Linux systems. A Singularity container bundles a primary application and all of its dependencies into a single image file, which can also include data, scripts, and other files if desired. In addition, Singularity containers have direct access to the Linux kernel on the host system (e.g., Discovery or Endeavour compute nodes), so there is no substantial performance penalty when using a container compared to using natively installed software on the host system.

With Singularity, you can:

• Install anything you want (based on any Linux operating system)
• Ease installation issues by using pre-built container images
• Ensure the same software stack is used among a research group
• Use the same software stack across Linux systems (e.g., any HPC center or cloud computing service)

Ansys

Description: The ANSYS suite of tools can be used to numerically simulate a wide range of structural and fluid dynamics issues encountered in several engineering, physics, medical, aerospace, and automotive sector applications.

Usage: Loading the ANSYS module module load ansys/2022r1 Launching the workbench is accomplished by: runwb2 The workbench provides access to Fluent, CFX, ICEM, Mechanical APDL/model, and many other languages and models. The appropriate GUIs can be launched outside of the workbench using fluent, cfx5pre, icemcfd, and launcher.

GROMACS

Description: GROningen MAchine for Chemical Simulations (GROMACS) is a free, open-source, molecular dynamics package. GROMACS can simulate the Newtonian equations of motion for systems with hundreds to millions of particles. GROMACS is primarily designed for biochemical molecules like proteins, lipids and nucleic acids that have a lot of complicated bonded interactions, but since GROMACS is extremely fast at calculating the nonbonded interactions (that usually dominate simulations), many groups are also using it for research on non-biological systems, e.g. polymers.

Usage: To load GROMACS software: module load GROMACS/2021.5-foss-2021b-CUDA-11.4.1 The GROMACS executable is either gmx or gmx mpi if an OpenMPI module is used. When you type gmx help commands, a list of gmx commands and their functions will be displayed.