Software 2
This page provides a comprehensive overview of the primary software utilized within Shabyt, along with guidelines on their operational workflows. It is essential to note that while this resource covers a significant portion of the software ecosystem within Shabyt, it is not an exhaustive list.
Environment Modules
Shabyt uses Environment modules to dynamically set up environments for different software. Module commands set, change, or delete environment variables that are needed for a particular software. The ‘module load‘ command will set PATH, LD_LIBRARY_PATH and other environment variables such that user may choose a desired version of applications or libraries more easily. More details can be found here.
| Command | Description |
|---|---|
| module avail | List of available software |
| module keyword [word] | Search for available modules matching the keyword |
| module spider [word] | Show the details of any modules matching the keyword |
| module whatis [module] | Show the short description about module |
| module load [package1] [package2] | Load the environment for the default version of the modulefile |
| module load [package]/[version] | Load the environment for the specified version of module |
| module unload [package1] [package2] | Unload previously loaded packages |
| module swap [moduleA] [moduleB] | Unload modulefile A and load modulefile B |
| module list | List any currently loaded module(s) |
| module purge | Unload all currently loaded modules |
Anaconda
Description: "Anaconda" (shortly "conda"), a Python package management, which helps you create an environment for many different versions of Python and package versions. Anaconda is also used to install, remove, and upgrade packages in your project environments.
Usage: Anaconda could be loaded by: module load Anaconda3/2022.05
Creating the Conda Environment: Every user can create their own environments, and packages that are shared with the system-wide environments will not be reinstalled or copied to your file store; instead, they will be symlinked. This reduces the amount of space required in your /home directory to install numerous Python environments. To build a pristine environment with only Python 3.9 and numpy, execute: conda create -n mynumpy1 python=3.9 numpy To alter an existing environment, such as one of the anaconda installations, you can clone it: conda create --clone mynumpy2 -n mynumpy3
Package Installation Within a Conda Environment: After creating your own environment, you can install more packages or other versions of existing programs. There are two ways to accomplish this: conda and pip. If a package is accessible through conda, it is strongly advised that you use it to install packages. Using conda, you can search for packages: conda search pandas then download the package by following these steps: conda install pandas When attempting to install packages outside of your environment, you will receive a permission denied message. If this occurs, create or activate an environment you own. If a package is not available via conda, you can search for and install it using pip: pip search colormath pip install colormath
Usage of conda Environments: Once the conda module has been loaded, the required conda environments must be loaded or created. See the conda manual for documentation on conda environments. You can load a conda environment with the following: source activate mycondaenv where mycondaenv is the environment's name; unload one with: source deactivate which returns you back to the base environment. You can list all the accessible environments by using: conda env list A set of anaconda environments is provided system-wide; these are installed with the anaconda version number in the environment name and are never updated. Therefore, they will provide a fixed base for alternative environments or for direct use.
Ansys
Description: The ANSYS suite of tools can be used to numerically simulate a wide range of structural and fluid dynamics issues encountered in several engineering, physics, medical, aerospace, and automotive sector applications.
Usage: Loading the ANSYS module module load ansys/2022r1 Launching the workbench is accomplished by: runwb2 The workbench provides access to Fluent, CFX, ICEM, Mechanical APDL/model, and many other languages and models. The appropriate GUIs can be launched outside of the workbench using fluent, cfx5pre, icemcfd, and launcher.
CUDA
Description: Nvidia created the parallel computing platform and programming model known as CUDA for use with its GPUs for general computing (graphics processing units). By utilizing the capability of GPUs for the parallelizable portion of the calculation, CUDA enables developers to accelerate computationally heavy applications.
Usage: module load CUDA/11.4.1 To check if CUDA has been loaded, type: nvcc --version
GCC
The GNU Compiler Collection, commonly known as GCC, is a set of compilers and development tools available for Linux, Windows, various BSDs, and a wide assortment of other operating systems. It includes support primarily for C and C++ and includes Objective-C, Ada, Go, Fortran, and D. The Free Software Foundation (FSF) wrote GCC and released it as completely free (as in libre) software.
GCC is a toolchain that compiles code, links it with any library dependencies, converts that code to assembly, and then prepares executable files. It follows the standard UNIX design philosophy of using simple tools that perform individual tasks well. The GCC development suite utilizes these discrete tools to compile software.
When you run GCC on a source code file, it first uses a preprocessor to include header files and discard comments. Next, it tokenizes the code, expands macros, detects any compile-time issues, then prepares it for compilation. It is then sent to the compiler, which creates syntax trees of the program’s objects and control flow and uses those to generate assembly code. The assembler then converts this code into the binary executable format of the system. Finally, the linker includes references to any external libraries as needed. The finished product is then executable on the target system.
GCC examples
Compiling a program with GCC can be a straightforward matter/
gcc hello.c -o hello
Running this command processes the hello.c file and generates a binary called “hello”. Additional parameters can be passed.
gcc hello.c -O3 -o hello
In this example, the optimization parameter is set to 3, leading to more optimized code generation. Additional libraries can be included as well.
gcc hello.c -lncurses -o hello
This example includes the ncurses library.
More complex compilations are managed by Makefiles and are invoked with the “make” command.
Additional information
GNU Compiler Collection official page: https://gcc.gnu.org/\
GROMACS
Description: GROMACS is a flexible package for performing molecular dynamics, simulating the Newtonian equations of motion for systems containing hundreds of thousands to millions of particles. It is intended for biochemical molecules, such as proteins, lipids, and nucleic acids, with complex bonded interactions. However, GROMACS is fast at calculating nonbonded interactions, so many groups use it for non-biological systems, like polymers.
Usage: To load GROMACS software: module load GROMACS/2021.5-foss-2021b-CUDA-11.4.1 The GROMACS executable is either gmx or gmx mpi if an OpenMPI module is used. When you type gmx help commands, a list of gmx commands and their functions will be displayed.
Batch jobs: Users are encouraged to create their own scripts for batch submissions. Below are examples of batch submission scripts.
Parallel MPI #!/bin/bash #SBATCH --job-name=gromacs #SBATCH --mail-user=<YOUR_NU_ID>@nu.edu.kz #SBATCH --mail-type=FAIL,BEGIN,END #SBATCH --output=gmx-%j.out #SBATCH --ntasks=2 #SBATCH --cpus-per-task=4 #SBATCH --ntasks-per-socket=1 #SBATCH --time=24:00:00 #SBATCH --mem-per-cpu=1gb module purge module load OpenMPI/4.1.1-GCC-11.2.0 module load GROMACS/2021.5-foss-2021b-CUDA-11.4.1 export OMP_NUM_THREADS=$SLURM_CPUS_PER_TASK srun --mpi=pmix_v3 gmx mdrun -ntomp ${SLURM_CPUS_PER_TASK} -s topol.tpr
Please ensure that you paste this code into the MediaWiki editor and make any necessary adjustments for formatting or links if needed.
Software Installation
Software installation on the Shabyt system follows specific criteria to ensure compatibility and effective utilization of resources. Users can request the installation of new software provided it meets the following conditions:
- Availability and Licensing: The software must be freely available or covered by a site license held by NU.
- Compatibility: It should be compatible with the existing operating system environment on Shabyt to ensure seamless integration and functionality.
- Resource Utilization: The software should be able to effectively utilize the resources available on Shabyt, optimizing performance and efficiency.
For guidance or support regarding the installation of new software packages, users can contact the Shabyt system administrators at hpcadmin@nu.edu.kz.
Additionally, software installations are prioritized as follows:
- EasyBuild Applications: Software that can be installed using the EasyBuild application is given first priority. A list of supported EasyBuild software can be found here.
- Critical User Group Applications: Applications essential for User Groups but not installable through EasyBuild are prioritized next, ensuring their availability and functionality for specific needs.
- Individual User Requests: Requests from individual users are processed after addressing the priorities mentioned above, ensuring a systematic approach to software installations based on need and compatibility.