3. Podman - Familiar Territory
3. Podman - Familiar Territory ๊ด๋ จ
The goal of this lab is to introduce you to Podman and some of the features that make it interesting. If you have ever used Docker, the basics should be pretty familiar. Lets start with some simple commands.
Pull an image:
podman pull ubi8
# Resolved "ubi8" as an alias (/etc/containers/registries.conf.d/001-rhel-shortnames.conf)
# Trying to pull registry.access.redhat.com/ubi8:latest...
# Getting image source signatures
# Checking if image destination supports signatures
# Copying blob 6c53be4efe39 done
# Copying config e8e5725e8a done
# Writing manifest to image destination
# Storing signatures
# e8e5725e8af3dfbee5236da434f811b3c5175d7057a279a7804bc34732ab35f9
List locally cached images:
podman images
# REPOSITORY TAG IMAGE ID CREATED SIZE
# registry.access.redhat.com/ubi8 latest e8e5725e8af3 6 weeks ago 215 MB
Start a container and run bash interactively in the local terminal. When ready, exit:
podman run -it ubi8 bash
exit
List running containers:
podman ps -a
# CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
# b723f1863d1a registry.access.redhat.com/ubi8:latest bash 16 seconds ago Exited (0) 11 seconds ago gallant_mahavira
Now, let's move on to some features that differentiates Podman from Docker. Specifically, let's cover the two most popular reasons - Podman runs with a daemon (daemonless) and without root (rootless). Podman does not have a daemon, it's an interactive command more like bash, and like bash it can be run as a regular user (aka. rootless).
The container host we are working with already has a user called RHEL, so let's switch over to it. Note, we set a couple of environment variables manually because we're using the switch user command. These would normally be set at login:
su - rhel
export XDG_RUNTIME_DIR=/home/rhel
Now, fire up a simple container in the background:
podman run -id ubi8 bash
# Resolved "ubi8" as an alias (/etc/containers/registries.conf.d/001-rhel-shortnames.conf)
# Trying to pull registry.access.redhat.com/ubi8:latest...
# Getting image source signatures
# Checking if image destination supports signatures
# Copying blob 6c53be4efe39 done
# Copying config e8e5725e8a done
# Writing manifest to image destination
# Storing signatures
# 7bb9972e6b51ba68432b6e00c857de926b1bb6d5411bd8fca865813ccbef37a9
Now, lets analyze a couple of interesting things that makes Podman different than Docker - it doesn't use a client server model, which is useful for wiring it into CI/CD systems, and other schedulers like Yarn:
Inspect the process tree on the system:
pstree -Slnc
You should see something similar to:
# โโconmonโโฌโ{conmon}
# โโbash(ipc,mnt,net,pid,uts)
There's no Podman process, which might be confusing. Lets explain this a bit. What many people don't know is that containers disconnect from Podman after they are started. Podman keeps track of meta-data in ~/.local/share/containers
(/var/lib/containers
is only used for containers started by root) which tracks which containers are created, running, and stopped (killed). The meta-data that Podman tracks is what enables a "podman ps" command to work.
In the case of Podman, containers disconnect from their parent processes so that they don't die when Podman exit exits. In the case of Docker and CRI-O which are daemons, containers disconnect from the parent process so that they don't die when the daemon is restarted. For Podman and CRI-O, there is utility which runs before runc called conmon (Container Monitor). The conmon utility disconnects the container from the engine by doing forking twice (called a double fork). That means, the execution chain looks something like this with Podman:
bash -> podman -> conmon -> conmon -> runc -> bash
Or like this with CRI-O:
systemd -> crio -> conmon -> conmon -> runc -> bash
Or like this with Docker engine:
systemd -> dockerd -> containerd -> docker-shim -> runc -> bash
Conmon is a very small C program that monitors the standard in, standard error, and standard out of the containerized process. The conmon utility and docker-shim both serve the same purpose. When the first conmon finishes calling the second, it exits. This disconnects the second conmon and all of its child processes from the container engine. The second conmon then inherits init system (systemd) as its new parent process. This daemonless and simplified model which Podman uses can be quite useful when wiring it into other larger systems, like CI/CD, scripts, etc.
Podman doesn't require a daemon and it doesn't require root. These two features really set Podman apart from Docker. Even when you use the Docker CLI as a user, it connects to a daemon running as root, so the user always has the ability escalate a process to root and do whatever they want on the system. Worse, it bypasses sudo rules so it's not easy to track down who did it.
Now, let's move on to some other really interesting features. Rootless containers use a kernel feature called User Namespaces. This maps the one or more user IDs in the container to one or more user IDs outside of the container. This includes the root user ID in the container as well as any others which might be used by programs like Nginx or Apache.
Podman makes it super easy to see this mapping. Start an nginx container to see the user and group mapping in action:
podman run -id registry.access.redhat.com/rhscl/nginx-114-rhel7 nginx -g 'daemon off;'
# Trying to pull registry.access.redhat.com/rhscl/nginx-114-rhel7:latest...
# Getting image source signatures
# Checking if image destination supports signatures
# Copying blob d6ebec67f811 done
# Copying blob 2b63304f1869 done
# Copying blob d518f7038c56 done
# Copying blob 3811c17826fe done
# Copying config 38ef2df4d9 done
# Writing manifest to image destination
# Storing signatures
# cf02ae16c1b538c0d6507f8b732f496605dcc67aba4aff51bdc7f5df1113e281
Now, execute the Podman bash command:
podman top -l args huser hgroup hpid user group pid seccomp label
# COMMAND HUSER HGROUP HPID USER GROUP PID SECCOMP LABEL
# nginx: master process nginx -g daemon off; 101000 1001 12450 default root 1 filter system_u:system_r:container_t:s0:c627,c786
# nginx: worker process 101000 1001 12470 default root 15 filter system_u:system_r:container_t:s0:c627,c786
# nginx: worker process 101000 1001 12471 default root 16 filter system_u:system_r:container_t:s0:c627,c786
# nginx: worker process 101000 1001 12472 default root 17 filter system_u:system_r:container_t:s0:c627,c786
# nginx: worker process 101000 1001 12473 default root 18 filter system_u:system_r:container_t:s0:c627,c786
Notice that the host user, group and process ID "in" the container all map to different and real IDs on the host system. The container thinks that nginx is running as the user "default" and the group "root" but really it's running as an arbitrary user and group. This user and group are selected from a range configured for the "rhel" user account on this system. This list can easily be inspected with the following commands:
cat /etc/subuid
# rhel:100000:65536
# bas:165536:65536
# mochtar:231072:65536
# ade:296608:65536
# arslan:362144:65536
# gke-930957db5604c7804fbd:427680:65536
# gke-f34473de869e40d6894d:493216:65536
The first number represents the starting user ID, and the second number represents the number of user IDs which can be used from the starting number. So, in this example, our RHEL user can use 65,535 user IDs starting with user ID 165536. The Podman bash command should show you that nginx is running in this range of UIDs.
The user ID mappings on your system might be different because shadow utilities (useradd
, usderdel
, usermod
, groupadd
, etc) automatically creates these mappings when a user is added. As a side note, if you've updated from an older version of RHEL, you might need to add entries to /etc/subuid
and /etc/subgid
manually.
OK, now stop all of the running containers. No more one liners like with Docker, it's just built in with Podman:
podman kill --all
# 7bb9972e6b51ba68432b6e00c857de926b1bb6d5411bd8fca865813ccbef37a9
# cf02ae16c1b538c0d6507f8b732f496605dcc67aba4aff51bdc7f5df1113e281
Remove all of the actively defined containers. It should be noted that this might be described as deleting the copy-on-write layer, config.json
(commonly referred to as the Config Bundle) as well as any state data (whether the container is defined, running, etc):
podman rm --all
# 7bb9972e6b51ba68432b6e00c857de926b1bb6d5411bd8fca865813ccbef37a9
# cf02ae16c1b538c0d6507f8b732f496605dcc67aba4aff51bdc7f5df1113e281
We can even delete all of the locally cached images with a single command:
podman rmi --all
# Untagged: registry.access.redhat.com/ubi8:latest
# Untagged: registry.access.redhat.com/rhscl/nginx-114-rhel7:latest
# Deleted: e8e5725e8af3dfbee5236da434f811b3c5175d7057a279a7804bc34732ab35f9
# Deleted: 38ef2df4d903a4a828cf42754cd15f9b31c96749e12d90698a5f6b7f9d47e526
The above commands show how easy and elegant Podman is to use. Podman is like a Chef's knife. It can be used for pretty much anything that you used Docker for, but let's move on to Builah and show some advanced use cases when building container images.