Apptainer: Difference between revisions

From NU HPC Wiki
Jump to navigation Jump to search
(Created page with "Singularity is an open-source application for creating and running software containers, designed primarily for high-performance computing on shared Linux-based computing clusters like CARC systems. Singularity containers provide a custom user space and enable portable, reproducible, stable, and secure software environments on Linux systems. A Singularity container bundles a primary application and all of its dependencies into a single image file, which can also include...")
 
No edit summary
 
(3 intermediate revisions by the same user not shown)
Line 1: Line 1:
Singularity is an open-source application for creating and running software containers, designed primarily for high-performance computing on shared Linux-based computing clusters like CARC systems.
Apptainer is the most widely used container system for HPC. It is a replacement (or next generation) for Singularity supported by the Linux Foundation.


Singularity containers provide a custom user space and enable portable, reproducible, stable, and secure software environments on Linux systems. A Singularity container bundles a primary application and all of its dependencies into a single image file, which can also include data, scripts, and other files if desired. In addition, Singularity containers have direct access to the Linux kernel on the host system (e.g., Discovery or Endeavour compute nodes), so there is no substantial performance penalty when using a container compared to using natively installed software on the host system.
Containers are a way to isolate your software and make it portable and reproducible. It is a valuable asset for reproducible science and, in addition, Its use is especially recommended when


With Singularity, you can:
# Users rely on “old” libraries (e.g., HDF4);
# Users need to update their library constantly;
# It also makes the user more independent of the cluster admins.


Install anything you want (based on any Linux operating system)
== Instructions ==
Ease installation issues by using pre-built container images
The steps to build a container always start with loading the apptainer module as follows
Ensure the same software stack is used among a research group
<code>module load apptainer</code>
Use the same software stack across Linux systems (e.g., any HPC center or cloud computing service)
For the next step, users have two options: 1) to just use a prebuilt image, or 2) to use a .def file (called recipe) that will contain the instructions to build the container. When using the latter option, we can build our container on top of another prebuilt image.
 
Building the container takes only one line of code, the structure (when building directly from a prebuilt image) is as follows
<code>apptainer build [OPTION] <name-for-the-container> <library>://<prebuilt-image></code>
In order to know all the available “options”, one can run <code>apptainer build --help</code>. For the <code><library></code> part, usually we rely on DockerHub (for more options see the apptainer documentation). In general, we are interested in building a container image that is a single <code>.sif</code> file that can be transferred between computers. However, there is an interesting option called <code>--sandbox</code>, which creates an expandable container image. Sandboxes will be useful when trying to build a custom image. However, this type of container has two side effects. First, these containers take up more space. Secondly, they are not reproducible.

Latest revision as of 06:55, 3 July 2024

Apptainer is the most widely used container system for HPC. It is a replacement (or next generation) for Singularity supported by the Linux Foundation.

Containers are a way to isolate your software and make it portable and reproducible. It is a valuable asset for reproducible science and, in addition, Its use is especially recommended when

  1. Users rely on “old” libraries (e.g., HDF4);
  2. Users need to update their library constantly;
  3. It also makes the user more independent of the cluster admins.

Instructions

The steps to build a container always start with loading the apptainer module as follows

module load apptainer

For the next step, users have two options: 1) to just use a prebuilt image, or 2) to use a .def file (called recipe) that will contain the instructions to build the container. When using the latter option, we can build our container on top of another prebuilt image.

Building the container takes only one line of code, the structure (when building directly from a prebuilt image) is as follows

apptainer build [OPTION] <name-for-the-container> <library>://<prebuilt-image>

In order to know all the available “options”, one can run apptainer build --help. For the <library> part, usually we rely on DockerHub (for more options see the apptainer documentation). In general, we are interested in building a container image that is a single .sif file that can be transferred between computers. However, there is an interesting option called --sandbox, which creates an expandable container image. Sandboxes will be useful when trying to build a custom image. However, this type of container has two side effects. First, these containers take up more space. Secondly, they are not reproducible.