Software Old

List of software packages currently available on Shabyt

  Note that there maybe different versions available for each of the
  listed modules. Type module avail in the terminal window to see the
  complete list of versions.
  Module Name Software Name Description
  [13]Anaconda Anaconda Anaconda is a distribution of the Python and R
  programming languages for scientific computing, that aims to simplify
  package management and deployment.[14]Product Page
  [15]ANSYS ANSYS ANSYS is a general-purpose, finite-element modeling
  package for numerically solving a wide variety of mechanical
  problems.[16]Product Page
  aocc AMD Optimizing C/C++ Compiler A set of LLVM compilers (clang,
  flang) and toolchain technologies for AMD x64 platform. [17]Product
  page
  aocl/aocc AMD Optimizing CPU Libraries for AOCC A set of numerical
  libraries tuned specifically for ADM EPYC processor family. It includes
  BLIS/BLAS, libFLAME, LAPACK, ScaLAPACK, FFTW, AMD Math Library, AMD
  Random Number Generator Library, and other components. [18]Product Page
  aocl/gcc9 AMD Optimizing CPU Libraries for GCC v9 A set of numerical
  libraries tuned specifically for ADM EPYC processor family. It includes
  BLIS/BLAS, libFLAME, LAPACK, ScaLAPACK, FFTW, AMD Math Library, AMD
  Random Number Generator Library, and other components. [19]Product Page
  cmake CMake A cross-platform open-source tool for build automation,
  testing, packaging and installation of software by using a
  compiler-independent method. [20]Product page
  [21]CUDA CUDA computing platform and API A parallel computing platform
  and application programming interface to use with Nvidia graphical
  processing units. [22]Product page
  dafoam DaFOAM Discrete Adjoint with OpenFOAM for High-fidelity
  Multidisciplinary Design Optimization.[23]Product Page
  easybuild EasyBuild EasyBuild is a tool that allows to perform
  automated and reproducible compilation and installation of
  software.[24]Product Page
  gaussian Gaussian 16 computational chemistry package A popular
  electronic structure package that implements a wide range of
  theoretical methods to investigate molecular properties and chemical
  problems. [25]Product Page
  gcc GNU Compiler Collection and Toolchain A set of standard compilers
  (gcc, g++, gfortran, and others). [26]Product Page
  [27]GROMACS Gromacs molecular dynamics package A popular molecular
  dynamics package for for simulations of large systems (e.g. proteins).
  [28]Product page
  intel Intel Parallel Studio XE Cluster Edition 2020 A set of compilers
  (icc, icpc, ifort, and others), numerical libraries (Intel MKL), MPI
  libraries, profiling and debugging tools tools (VTune, Advisor,
  Inspector), Integrated Performance Primitives (IPP), Data Analytics
  Acceleration Library (DAAL), Intel Distribution of Python and other
  components. [29]Product Page
  lammps LAMMPS molecular dynamics package LAMMPS (Large-scale
  Atomic/Molecular Massively Parallel Simulator) is a molecular dynamics
  software package developed at Sandia National Laboratories. [30]Product
  page
  matlab MATLAB MATLAB is a proprietary multi-paradigm programming
  language and numeric computing environment developed by MathWorks.
  [31]Product Page
  nvhpc-byo-compiler NVIDIA ����HPC Software Development Kit (BYOC – Bring
  Your Own Compiler) Same as nvhpc but does not add PGI-compiler or MPI
  to the environment. [32]Product Page
  nvhpc-nompi NVIDIA HPC Software Development Kit (no MPI) Same as nvhpc
  but without adding any MPI installation to the environment. [33]Product
  Page
  nvhpc NVIDIA HPC Software Development A suite of PGI compilers (nvcc,
  nvc++, nvfortran), libraries and tools for HPC. [34]Product Page
  openfoam OpenFOAM OpenFOAM (for "Open-source Field Operation And
  Manipulation") is a C++ toolbox for the development of customized
  numerical solvers, and pre-/post-processing utilities for the solution
  of continuum mechanics problems, most prominently including
  computational fluid dynamics (CFD). [35]Product Page
  openmpi/aocc Open MPI module for AOCC An industry standard
  implementation of MPI (Message Passing Interface) developed and
  maintained by a consortium of academic, research, and industry
  partners. [36]Product Page
  openmpi/gcc4 Open MPI module for GCC v4 An industry standard
  implementation of MPI (Message Passing Interface) developed and
  maintained by a consortium of academic, research, and industry
  partners. [37]Product Page
  openmpi/gcc9 Open MPI module for GCC v9 An industry standard
  implementation of MPI (Message Passing Interface) developed and
  maintained by a consortium of academic, research, and industry
  partners. [38]Product Page
  openmpi/gcc10 Open MPI module for GCC v10 An industry standard
  implementation of MPI (Message Passing Interface) developed and
  maintained by a consortium of academic, research, and industry
  partners. [39]Product Page
  openmpi/gcc11 Open MPI module for GCC v11 An industry standard
  implementation of MPI (Message Passing Interface) developed and
  maintained by a consortium of academic, research, and industry
  partners. [40]Product Page
  openmpi/intel Open MPI module for Intel compiler suite An industry
  standard implementation of MPI (Message Passing Interface) developed
  and maintained by a consortium of academic, research, and industry
  partners. [41]Product Page
  quantumespresso QuantumESPRESSO Quantum ESPRESSO is a suite for
  first-principles electronic-structure calculations and materials
  modeling.[42]Product Page
  R R R is a programming language for statistical computing and graphics
  supported by the R Core Team and the R Foundation for Statistical
  Computing.[43]Product Page
  singularity Singularity Singularity is a container solution created by
  necessity for scientific and application driven workloads.[44]Product
  Page
  tensorflow TensorFlow TensorFlow is an end-to-end open source platform
  for machine learning.[45]Product Page
  wps WPS The WRF Pre-Processing System (WPS) is a collection
  of Fortran and C programs that provides data used as
  input to the real.exe program. [46]Product Page
  wrf WRF The Weather Research and Forecasting (WRF) Model is a
  next-generation mesoscale numerical weather prediction system designed
  for both atmospheric research and operational forecasting
  applications.[47]Product Page
  New software
  New software can be installed system-wide at the request of users,
  provided it meets the following criteria:
    * It is either freely available or NU has a site license for it
    * It is compatible with the existing OS environment on Shabyt
    * It can utilize resources available on Shabyt effectively
  For assistance regarding new software packages please contact Shabyt
  system administrators at [48]hpcadmin@nu.edu.kz
  Anaconda
  Description:
  "Anaconda" (shortly “conda”), a Python package management, permits the
  creation of "environments," which are sets of modifiable packages. It
  accomplishes this by placing them in your residence. This page will
  take you through conda loading, environment creation, and modification
  so you may install and utilize any Python packages you require.
  Usage:
  Anaconda could be loaded by:
  module load Anaconda3/2022.���	05
   Creating the Conda Environment
  Every user can create their own environments, and packages that are
  shared with the system-wide environments will not be reinstalled or
  copied to your file store; instead, they will be symlinked. This
  reduces the amount of space required in your /home directory to install
  numerous Python environments.
  To build a pristine environment with only Python 3.9 and numpy,
  execute:
  conda create -n mynumpy1 python=3.9 numpy
  This will download the most recent version of Python 3.9 and numpy and
  build a mynumpy1 environment. Any version of Python or package list may
  be supplied:
  conda create -n mynumpy2 python=3.9 numpy=1.15.2 scipy
  To alter an existing environment, such as one of the anaconda
  installations, you can clone it:
  conda create --clone mynumpy2 -n mynumpy3
  This will create an environment called mynumpy3 which has all the same
  conda packages as the mynumpy2 environment.
  Package Installation Within a Conda Environment
  After creating your own environment, you can install more packages or
  other versions of existing programs. There are two ways to accomplish
  this: conda and pip. If a package is accessible through conda, it is
  strongly advised that you use it to install packages. Using conda, you
  can search for packages:
  conda search pandas
  then download the package by following these steps:
  conda install pandas
  When attempting to install packages outside of your environment, you
  will receive a permission denied message. If this occurs, create, or
  activate an environment you own. If a package is not available via
  conda, you can search for and install it using pip:
  pip search colormath
  pip install colormath
  Usage of conda Environments
  Once the conda module has been loaded, the required conda environments
  must be loaded or created. See the conda manual for documentation on
  conda environments. You can load a conda environment with the
  following:
  source activate mycondaenv
  where mycondaenv is the environment's name, unload one with:
  source deactivate
  which returns you back to the base environment. You can list all the
  accessible environments by using:
  conda env list
  A set of anaconda environments are provided system-wide; these are
  installed with the anaconda version number in the environment name and
  are never updated. Therefore, they will provide a fixed base for
  alternative environments or for direct use.
  Using Conda in conjunction with the SLURM scheduler
  Using the Anaconda batch mode
  To submit jobs to the Slurm job scheduler, you must run your primary
  application in batch mode from within your Conda environment. There are
  several stages to take:
  Create a script for application
  Create a Slurm script job to execute the application script
  sbatch is used to submit the job script to the task scheduler.
  Your application script should provide the necessary sequence of
  commands for your analysis. A Slurm job script is a Bash shell script
  of a particular type that the Slurm job scheduler recognizes as a job.
  Create a batch job submission script like the following and name it
  myscript.slurm:
  #!/bin/bash
  #SBATCH --partition=NVIDIA
  #SBATCH --nodes=1
  #SBATCH --ntasks=1
  #SBATCH --cpu-per-task=8
  #SBATCH --mem=16GB
  #SBATCH --time=1:00:00
  module purge
  eval $(conda shell.bash hook)”
  conda activate myenvironment
  python script.py
  The following describes each line:
  Command or Slurm option
  Description
  #!/bin/bash
  Use BASH to execute the script
  #SBATCH
  Syntax that allows SLURM to read your requests (ignored by BASH)
  --partition=NVIDIA
  Submit job to the NVIDIA partition
  --nodes=1
  Use only one compute node
  --ntasks=1
  Run only one task
  --cpus-per-task=8
  Reserve 8 CPUs for user
  --mem=16GB
  Research 16GB of RAM
  --module purge
  Purge or clear environment modules
  --time=1:00:00
  Reserve resource for an hour
  eval $(conda shell.bash hook) Initi����alize the shell to use Conda
  conda activate myenvironment Activate your Conda environment, i.e.,
  myenvironment
  python script.py Use python to run script.py
  Be sure to alter the requested resources based on your needs, but keep
  in mind that requesting fewer resources will reduce the waiting time
  for your work. To fully exploit the resources, particularly the amount
  of cores, it may be necessary to modify your application script.  You
  can build application scripts and job scripts on your local machine and
  then move them to the cluster, or you can develop them directly on the
  cluster using one of the various text editor modules (e.g., nano,
  micro, vim). Submit the job to the job scheduler using the
  Slurm's sbatch command:
  sbatch myscript.slurm
  To determine the status of your position, enter:
  myqueue
  If there is no job status listed, then the job has been finished. The
  job's findings will be logged and, by default, stored to a plain-text
  file of the following format:
  slurm-<jobid>.outin
  the identical directory from which the job script was submitted. To
  access this file's contents, enter:
  less slurm-<jobid>.out
  then press q to close the viewer.
  CUDA
  Description:
  Nvidia created the parallel computing platform and programming model
  known as CUDA for use with its own GPUs for general computing (graphics
  processing units). By utilizing the capability of GPUs for the
  parallelizable portion of the calculation, CUDA enables developers to
  accelerate computationally heavy applications.
  Usage:
  module load CUDA/11.4.1
  to check if CUDA has been loaded, type:
  nvcc --version
  ANSYS
  Description:
  The ANSYS suite of tools can be used to numerically simulate a wide
  range of structural and fluid dynamics issues encountered in several
  engineering, physics, medical, aerospace, and automotive sector
  applications.
  Usage:
  Loading the ANSYS module
  module load ansys/2022r1
  launching the workbench is accomplished by:
  runwb2
  The workbench provides access to Fluent, CFX, ICEM, Mechanical
  APDL/model, and many other languages and models. The appropriate GUIs
  can be launched outside of the workbench using fluent, cfx5pre,
  icemcfd, and launcher.
  GROMACS
  Description: GROMACS is a flexible package for performing molecular
  dynamics, i.e., simulating the Newtonian equations of motion for
  systems containing hundreds of thousands to millions of particles. It
  is primarily intended for biochemical molecules, such as proteins,
  lipids, and nucleic acids, that have many complex bonded interactions.
  However, since GROMACS is extremely fast at calculating the nonbonded
  interactions (which typically dominate simulations), many groups use it
  for research on non-biological systems, such as polymers.
  Usage:
  To load GROMACS software:
  module load GROMACS/2021.5-foss-2021b-CUDA-11.4.1
  The GROMACS executable is either gmx or gmx mpi if an OpenMPI module is
  being used. When you type gmx help commands, a list of gmx commands and
  their functions will be displayed.
  Batch jobs
  Users are encouraged to create their own scripts for batch submissions.
  The following are examples of batch submission scripts, my job.sh, that
  are submitted to the queue by typing qsub my job.sh.
  In addition to GPU acceleration, GROMACS can operate in two different
  parallel environments, smp and mpi. Below are examples of these types
  of batch submissions.
  Parallel MPI
  #!/bin/bash
  #SBATCH --job-name=gromacs
  #SBATCH --mail-user=<YOUR_NU_ID>@nu.edu.kz
  #SBATCH --mail-type=FAIL,BEGIN,END
  #SBATCH --output=gmx-%j.out
  #SBATCH --ntasks=2
  #SBATCH --cpus-per-task=4
  #SBATCH --ntasks-per-socket=1
  #SBATCH --time=24:00:00
  #SBATCH --mem-per-cpu=1gb
  module purge
  module load OpenMPI/4.1.1-GCC-11.2.0
  module load GROMACS/2021.5-foss-2021b-CUDA-11.4.1
  export OMP_NUM_THREADS=$SLURM_CPUS_PER_TASK
  srun --mpi=pmix_v3 gmx mdrun -ntomp ${SLURM_CPUS_PER_TASK} -s topol.tpr����neral NU Policies that
  govern the use of shared use facilities.
  Please note that the interactive login node (mgmt01) is designated to
  tasks such as user file management, code compilation, and job
  submission. It is NOT designated for long calculations and heavy data
  processing. Simply put, users should not run production calculations on
  it. Instead, all heavy computational tasks should be sent to a batch
  queue. Users who repeatedly abuse this rule of using the login node
  propely may be subject to account suspension.

Maintenance

  Most maintenance work is performed by the HPC team in the background
  and does not interfere with user activities. However, there may be
  situations when an interruption of the service is required for
  system-wide maintenance. This will be planned ahead of time and
  scheduled with at least several days of advance notice. Batch queues
  may be temporarily modified prior to scheduled downtimes to hold jobs
  that require more time than remains before the shutdown.

User groups

  Any new account on NU HPC systems (e.g. for a graduate student or a
  postdoc who does computational research) requires sponsorship/approval
  by the PI of a research project. Thus, all users are structured into
  two types of groups. The first type is a group of an individual PI.
  Then the PIs’ groups are structured into three larger groups: NU group,
  HPCNC Research Cluster Group (this group is the principal host of
  Shabyt), and External Users group. Please note that the priorities of
  the jobs submitted for execution on Shabyt may depend on multiple
  factors, such as the number of jobs belonging to an individual PI group
  currently running and/or recently completed on the system, which of the
  three larger groups the user belongs to, the amount of resources
  requested by the job, and others. The algorithm for setting job
  priorities is subject to change and will be guided by the NU HPC
  Committee.

Storage quotas

  Current default storage quota for users’ home directories is set to 100
  GB. If any individual user requires more storage for his/her work, it
  can be allocated through a special request to the HPC admins. For
  particularly large, multi-terabyte storage needs Shabyt has an HDD
  array with the total capacity of 120 TB.

Data backup

  Please be advised that users take full responsibility for the integrity
  and safety of their data stored on Shabyt. While Shabyt features
  enterprise level hardware, failures are still a possibility, especially
  given no redundancy in the underlying storage systems that are designed
  for high throughput. Currently Shabyt does not have any automatic
  backup of users’ data. While this may change in the future as the HPC
  team continues to configure the system, at this time users are strongly
  advised to perform regular back up of any important data they may have
  stored in their home directories.

Queues and the number of jobs

  Currently, Shabyt has two partitions for user jobs. While at this time,
  when the system is still being configured and fine-tuned, there is no
  hardcoded limit on the number of jobs by any individual user, it will
  likely change in the near future.

Acknowledgment

  If the computational resources provided by NU HPC facilities were an
  important asset in your work resulting in a publication, we will
  greatly appreciate your acknowledgment. We suggest the following form
  of the acknowledgment statement at the end of your publication:
  “The authors are grateful to Nazarbayev University Research
  Computing for providing computational resources for this work.”
  If you wish to additionally acknowledge an individual from NU Research
  Computing team who spent considerable amount of time assisting you with
  your calculations you are also encouraged to do so, e.g.
  “We thank [assistant’s name] for [his/her/theirs] help with [your task
  accomplished], which was made possible through Nazarbayev University