Singularity

Singularity as an alternative container tool#

Singularity is a container software alternative to Docker. It was originally developed by researchers at Lawrence Berkeley National Laboratory with focus on security, scientific software, and HPC clusters. One of the ways in which Singularity is more suitable for HPC is that it very actively restricts permissions so that you do not gain access to additional resources while inside the container. Inside a Singularity container, you are the same user as you are on the host system.

Here we give a brief introduction to Singularity and specifically how it can be used on HPC clusters.

If you want to read more, here are some additional resources: Singularity docs

The basics#

Build a container

singularity build singularity.sif singularity.def

Run a command from a container

singularity run singularity.sif echo toto

Executing a script inside the container

singularity exec singularity.sif

Use a container interactively

singularity run singularity.sif

• Unlike Docker, Singularity attempts to map parts of your local file system to the image ($PWD (the current working directory), /home and few other things).

• In addition to these defaults, you can specify your own directories to bind mount using the --bind or -B option in Singularity commands or using SINGULARITY_BINDPATH environment variable.

• Singularity, unlike Docker, stores images as single files. The image file is self-contained (no shared layers) and can be moved around and shared like any other file.

Build a Singularity image#

• Building an image means creating a container that will be in the Singularity Image Format (.sif file).

• It can de done from a Singularity Definition file .def or by converting an existing docker container.

• Singularity images are read-only, unless you use the sandbox feature.

When using sandbox mode, the container image is created as a writable directory that represents the container's internal file system. You can then install software interactively using the Singularity shell.

Using sandbox mode can be more intuitive and faster than using definition files, especially if you need to make adjustments (e.g., finding the right library versions). However, containers created this way are not easily reproducible. It's a good practice to keep track of the commands you use and write a definition file afterward.

from a Singularity Definition file#

• To build a Singularity image we need a Singularity Definition file .def.

• command:

singularity build singularity.sif singularity.def

• The resulting .sif image file can be moved around and shared like any other file.

Let's go through an example of .def file (below) and discuss the different steps and what they do.

BootStrap: docker
From: ubuntu:20.04

# This section will label the container
%labels
    Author John Doe
    Version 1.0

# help message that users can view by running: singularity help <container.sif>s
%help
    This is an example Singularity definition file that sets up a basic Ubuntu container and installs some common packages.

# The post section is used to install packages and configure the environment
%post
    # Update package lists
    apt-get update
    # Install necessary packages
    apt-get install -y wget git python3 python3-pip
    # Clean up
    apt-get clean
    rm -rf /var/lib/apt/lists/*

# Define environment variables for use in the container
%environment
    # Set environment variables
    export PATH=/usr/local/bin:$PATH
    export MY_VAR="Hello from Singularity!"

# The runscript will run when the container is executed
%runscript
    # This is the default behavior when the container is run
    echo "This container is running!"
    echo "The value of MY_VAR is: $MY_VAR"
    # pass any additional arguments provided to the container when it’s executed
    exec "$@"

# A test section
%test
    # Test that Python is installed
    python3 --version

More information how to build a Singularity images from scratch can be found in here.

** Docker vs Singularity **

Converting Docker images to Singularity files#

Singularity can convert Docker images to the Singularity Image Format (SIF).
This is great if there's a Docker image that you want to use on an HPC cluster where you cannot use Docker.

Here an example how to retrieve and convert a container from DockerHub into a Singularity Image:

singularity pull docker://condaforge/miniforge3:24.7.1-2

This should result in a file called miniforge3_24.7.1-2.sif.

Running a singularity image#

• Run the image

The singularity run command executes the default script defined in the Singularity image (via the %runscript section in the definition file):

singularity run ubuntu_20.04.sif

• Execute Commands in the Container

To run specific commands inside the container, use the exec command:

singularity exec ubuntu_20.04.sif echo "Hello from the container"

• Shell Access

Run bash interactive commands :

singularity exec ubuntu_20.04.sif /bin/bash

You can also open an interactive shell inside the container with shell:

singularity shell ubuntu_20.04.sif

Inspecting a Singularity container#

• View Metadata

To see the metadata (labels, help, etc.) inside the container, use:

singularity inspect ubuntu_20.04.sif

• Get Help

singularity help ubuntu_20.04.sif

Binding Directories#

By default Singularity bind mounts:

• /home  
• /tmp and /var/tmp  
• /proc  
• /sys  
• /dev  
• /etc/passwd, /etc/group, /etc/resolv.conf  
• $PWD (the current working directory)

To access other files/directories from your host system within the container, you can bind mount directories using the -B option:

singularity exec -B /host/directory:/container/directory ubuntu_20.04.sif ls /container/directory

Containers at different platforms#

A common problem with Singularity is that you can only create local builds if you are working on a Linux system, as local builds for MacOS and Windows are currently not supported. This means that you might favour using Docker instead of Singularity, but what happens when you need to use a HPC cluster such as HPC? Docker won't work there, as it requires root privileges, so Singularity is the only solution.