Pulling Containers
Pulling pre-built container images is often the easiest and quickest way to run your desired software enviroment. In this tutorial, we will pull a container from Docker Hub. DockerHub is a cloud-based registry that provides access to a large repository of pre-built container images. Other container registries include NVIDIA NGC, Biocontainers, Sylabs.io, and Quay.io. For more information on accessible container registries, check out the containers documentation page HERE.
Set Up#
To start, log into Hyak and navigate away from your home directory to a directory where you have access to storage. For this tutorial, we will work these exercises using a directory in /gscratch/scrubbed/. You may choose a working directory under another path on the filesystem, but we recommend not performing these exercises in your Home directory due to storage limitations. Learn more about storage on Hyak HERE.
Navigate to /gscratch/scrubbed/
If you have not already, make a directory with your UW NetID.
Next, ensure that the tutorial materials are copied to your working directory. In this tutorial, your copy of the basics directory will be our working directory:
The program klone uses for containers is Apptainer. When pulling a Docker container, Apptainer is able to convert the Docker container into an Apptainer container. However, to use Apptainer, you must be on a compute node:
Remember to use hyakalloc to see all of your available resources. If you are a demo account user, please use the ckpt-all partition as shown above.#
Pulling a Container to Hyak from Docker Hub#
For this exercise, we will pull a Python container from Docker Hub. Follow this link to open Docker Hub and search for Python. Choose the first search result with the green badge symbol next to it. This green symbol indicates that it is an official docker image.
Next, click on "Tags" tab next to the "Overview" tab. Notice how you can search for previous versions of Python here. In this tutorial, we will download the most recent verison of Python (as of Fall 2024), which is installed in a container running Ubuntu's slim-bullseye release. Copy the code from the code box for the 3.9.20-bullseye image and paste it into your terminal.
Omit docker pull and replace it with apptainer pull docker:// so Apptainer pulls the container image from Docker Hub and converts it into an Apptainer container. Run this code to pull and build the container.
The build should last 30 seconds to 1 minute. During the pull and build, you will see messages from Apptainer on the progress of the command.
Once that is complete, check to see if it was sucessfully built. The container will have a file extension .sif.
You are now able to open a shell inside the container where you can run Python:
The command prompt will change and now start with Apptainer>. This is how you know that you are now inside the Apptainer container. Input python into the terminal to open the container's Python shell:
>>> should appear on the command prompt, indiciating that you are now able to use Python interactively. To exit Python, use Ctrl+D.
To exit the container write exit
note
The above tutorial is useful if you want to use the most recent verison of Python. However, there is a native version of Python installed on Hyak. Check what version of Python this is with:
You can check the version of Python in the container with the following command:
apptainer exec rather than apptainer shell sends a command to be executed inside the container, rather than first opening a shell into the container.
Binding The Filesystem#
Recall that Apptainer containers are by default read-only. This means that under default parameters you cannot store files within the container and the container is isolated so that it cannot access any files from outside the container. To demonstrate this, try to execute ls from within the container, and you will see none of your files and directories on Hyak will appear.
Open a shell into the container once more
Try ls to list the directory.
This isn't very practical. We will usually have more files outside of the container upon which we want to execute commands. To make the Hyak filesystem accessible to the container, you must bind your filesystem to your container. In this section, we will bind the Hyak filesystem and run a simple Python script in the container. If you have not already, exit Apptainer with exit. Using the nano text editor, let's create our Python script:
Try to run the pi.py script without binding the filesystem.
To access the pi.py script and execute it, the file system much be bound to the container. To bind the filesystem to the container AND run pi.py, use the following command:
You can also use apptainer shell to open a shell into the container and bind the filesystem so that the files are accessible and visible with ls.
Now with ls you can list the contents of your working directory.
Now you can now run pi.py inside the shell:
In the next section, we'll learn how to build custom containers.