Software 2

New Software

New software can be installed system-wide at the request of users, provided it meets the following criteria:

• It is either freely available or NU has a site license for it.
• It is compatible with the existing OS environment on Shabyt.
• It can utilize resources available on Shabyt effectively.

For assistance regarding new software packages, please contact Shabyt system administrators at hpcadmin@nu.edu.kz.

Software Priorities

Software applications are installed in accordance with priorities.

• Software which can be installed via EasyBuild application falls under this category. List of supported EasyBuild softwares you can find here: EasyBuild
• Applications that are crucial for User Groups but cannot be installed through EasyBuild fall under this priority.
• Individual user requests. Such requests will be processed only after other priorities have been fulfilled.

Software List

Name	Description	Versions
AMD-uProf	AMD uProf is a performance analysis tool for applications running on Windows, Linux & FreeBSD operating systems. It allows developers to better understand the runtime performance of their application and to identify ways to improve its performance.	AMD-uProf/3.5.671
AOCC	AOCC is a high-performance x86 CPU compiler for C, C++, and Fortran programming languages	AOCC/4.0.0
Anaconda3	Anaconda: Python Data Science Platform for Python 3	Anaconda3/2022.05
Automake	Automake – make file builder part of autotools	Automake/1.16.1 Automake/1.16.3 Automake/1.16.4 Automake/1.16.5 (D)
Bison	Bison is a general-purpose parser generator that converts an annotated context-free grammar into a deterministic LR or generalized LR (GLR) parser employing LALR(1) parser tables.	Bison/3.5.3 Bison/3.7.1 Bison/3.7.6 Bison/3.8.2 (D)
CMake	CMake is the standard for building C++ code	CMake/3.20.1 CMake/3.21.1 CMake/3.22.1 CMake/3.23.1 (D)
CUDA	CUDA (formerly Compute Unified Device Architecture) is a parallel computing platform and programming model created by NVIDIA and implemented by the graphics processing units (GPUs) that they produce. CUDA gives developers access to the virtual instruction set and memory of the parallel computational elements in CUDA GPUs.	CUDA/11.4.1 CUDA/11.7.0 (D)
Doxygen	Doxygen is a widely-used documentation generator tool in software development. It automates the generation of documentation from source code comments, parsing information about classes, functions, and variables to produce output in formats like HTML and PDF.	Doxygen/1.9.1 Doxygen/1.9.4 (D)
Eigen	Eigen is a C++ template library for linear algebra: matrices, vectors, numerical solvers, and related algorithms.	Eigen/3.3.9 Eigen/3.4.0 (D)
FFTW	FFTW is a C subroutine library for computing the discrete Fourier transform (DFT) in one or more dimensions, of arbitrary input size, and of both real and complex data (as well as of even/odd data, i.e. the discrete cosine/sine transforms or DCT/DST)	FFTW/3.3.9 FFTW/3.3.10 (D)
FFmpeg	FFmpeg is the multimedia framework, able to decode, encode, transcode, mux, demux, stream, filter and play audio stream and video.	FFmpeg/4.3.2 FFmpeg/4.4.2 (D)
FLAC	FLAC is an audio format similar to MP3, but lossless, meaning that audio is compressed in FLAC without any loss in quality.	FLAC/1.3.4
GCC	The GNU Compiler Collection includes front ends for C, C++, Objective-C, Fortran, Ada, Go, and D, as well as libraries for these languages.	GCC/9.3.0 GCC/9.4.0 GCC/10.3.0 GCC/11.2.0 GCC/11.3.0 GCC/12.2.0 (D)
GDAL	GDAL is a translator library for raster and vector geospatial data formats that is released under an MIT style Open Source License by the Open Source Geospatial Foundation. As a library, it presents a single raster abstract data model and single vector abstract data model to the calling application for all supported formats. It also comes with a variety of useful command line utilities for data translation and processing.	GDAL/3.5.0-foss-2022a
GEOS	GEOS is a C/C++ library for computational geometry with a focus on algorithms used in geographic information systems (GIS) software.	GEOS/3.10.3
GLPK	The GLPK (GNU Linear Programming Kit) package is intended for solving large-scale linear programming (LP), mixed integer programming (MIP) problems.	GLPK/5.0
GMP	GMP is a free library for arbitrary precision arithmetic, operating on signed integers, rational numbers, and floating-point numbers.	GMP/6.2.1
GObject-Introspection	GObject introspection is a middleware layer between C libraries (using GObject) and language bindings. The C library can be scanned at compile time and generate metadata files, in addition to the actual native C library.	GObject-Introspection/1.68.0 GObject-Introspection/1.72.0 (D)
GSL	The GNU Scientific Library (GSL) is a numerical library for C and C++ programmers. It is free software under the GNU General Public License.	GSL/2.7
GTK2	The GTK+ 2 package contains libraries used for creating graphical user interfaces for applications.	GTK2/2.24.33
Ghostscript
HDF
HDF5
HarfBuzz
Hypre
ICU
ImageMagick
Jesper
Java
KaHIP
LAMMPS
LLVM
LibTIFF
LittleCMS
Lua
M4

Anaconda

Description: "Anaconda" (shortly "conda"), a Python package management, permits the creation of "environments," which are sets of modifiable packages. It accomplishes this by placing them in your residence. This page will take you through conda loading, environment creation, and modification so you may install and utilize any Python packages you require.

Usage: Anaconda could be loaded by: module load Anaconda3/2022.05

Creating the Conda Environment: Every user can create their own environments, and packages that are shared with the system-wide environments will not be reinstalled or copied to your file store; instead, they will be symlinked. This reduces the amount of space required in your /home directory to install numerous Python environments. To build a pristine environment with only Python 3.9 and numpy, execute: conda create -n mynumpy1 python=3.9 numpy To alter an existing environment, such as one of the anaconda installations, you can clone it: conda create --clone mynumpy2 -n mynumpy3

Package Installation Within a Conda Environment: After creating your own environment, you can install more packages or other versions of existing programs. There are two ways to accomplish this: conda and pip. If a package is accessible through conda, it is strongly advised that you use it to install packages. Using conda, you can search for packages: conda search pandas then download the package by following these steps: conda install pandas When attempting to install packages outside of your environment, you will receive a permission denied message. If this occurs, create or activate an environment you own. If a package is not available via conda, you can search for and install it using pip: pip search colormath pip install colormath

Usage of conda Environments: Once the conda module has been loaded, the required conda environments must be loaded or created. See the conda manual for documentation on conda environments. You can load a conda environment with the following: source activate mycondaenv where mycondaenv is the environment's name; unload one with: source deactivate which returns you back to the base environment. You can list all the accessible environments by using: conda env list A set of anaconda environments is provided system-wide; these are installed with the anaconda version number in the environment name and are never updated. Therefore, they will provide a fixed base for alternative environments or for direct use.

Using Conda in conjunction with the SLURM scheduler: Using the Anaconda batch mode To submit jobs to the Slurm job scheduler, you must run your primary application in batch mode from within your Conda environment. There are several stages to take: Create a script for application Create a Slurm script job to execute the application script sbatch is used to submit the job script to the task scheduler. Your application script should provide the necessary sequence of commands for your analysis. A Slurm job script is a Bash shell script of a particular type that the Slurm job scheduler recognizes as a job. Create a batch job submission script like the following and name it myscript.slurm: <clippy show="true">

1. !/bin/bash
2. SBATCH --partition=NVIDIA
3. SBATCH --nodes=1
4. SBATCH --ntasks=1
5. SBATCH --cpu-per-task=8
6. SBATCH --mem=16GB
7. SBATCH --time=1:00:00

module purge eval $(conda shell.bash hook) conda activate myenvironment python script.py </clippy> The following describes each line: Command or Slurm option Description

1. !/bin/bash

Use BASH to execute the script

1. SBATCH

Syntax that allows SLURM to read your requests (ignored by BASH) --partition=NVIDIA Submit job to the NVIDIA partition --nodes=1 Use only one compute node --ntasks=1 Run only one task --cpus-per-task=8 Reserve 8 CPUs for the user --mem=16GB Reserve 16GB of RAM --module purge Purge or clear environment modules --time=1:00:00 Reserve resources for an hour eval $(conda shell.bash hook) Initialize the shell to use Conda conda activate myenvironment Activate your Conda environment, i.e., myenvironment python script.py Use python to run script.py Be sure to alter the requested resources based on your needs, but keep in mind that requesting fewer resources will reduce the waiting time for your work. To fully exploit the resources, particularly the amount of cores, it may be necessary to modify your application script. You can build application scripts and job scripts on your local machine and then move them to the cluster, or you can develop them directly on the cluster using one of the various text editor modules (e.g., nano, micro, vim). Submit the job to the job scheduler using the Slurm's sbatch command: sbatch myscript.slurm To determine the status of your position, enter: myqueue If there is no job status listed, then the job has been finished. The job's findings will be logged and, by default, stored in a plain-text file of the following format: slurm-<jobid>.outin in the identical directory from which the job script was submitted. To access this file's contents, enter: less slurm-<jobid>.out then press q to close the viewer.