Containers are Linux: namespaces, cgroups and Docker

A container is a process

Everything in this course applies inside containers, because a container is a Linux process. The kernel has no "container" object. It has two features that, used together, make a process look like it has a machine to itself:

⤢ View

Namespaces: what it can see. cgroups: what it can use. One shared kernel underneath.© Diagram: DevOps Academy

Close

Namespaces: what a process can see

Each namespace gives a process its own view of one part of the system:

$ ls /proc/self/ns                # every process has one of each
cgroup  ipc  mnt  net  pid  pid_for_children  time  time_for_children  user  uts

You can make namespaces yourself, without root, using unshare (-r maps you to root inside a new user namespace):

$ unshare -r -u sh -c 'hostname box1; hostname'     # new UTS namespace: own hostname
box1
$ hostname                                          # the host is untouched
Elonerajeev
$ unshare -r -p -f --mount-proc ps -ef              # new PID namespace
UID          PID    PPID  C STIME TTY          TIME CMD
root           1       0  0 05:40 ?        00:00:00 ps -ef

Inside the new PID namespace, ps is PID 1 and sees nothing else. That's exactly why docker exec … ps shows only the container's processes. nsenter does the reverse: it runs a command inside an existing process's namespaces, which is how you debug a container that has no shell.

cgroups: what a process can use

Control groups put processes in a tree and limit each branch: memory, CPU, number of processes, disk I/O. Modern distributions use cgroups v2, one tree under /sys/fs/cgroup:

$ stat -fc %T /sys/fs/cgroup              # cgroup2fs means v2
cgroup2fs
$ cat /proc/self/cgroup                   # which cgroup this shell is in
$ systemd-cgls --no-pager | head          # the whole tree: systemd organises it
$ systemd-cgtop                           # live use per cgroup, like top

systemd puts every service in its own cgroup, which is why systemctl status shows a service's memory. You can limit any command or service the same way a container runtime does:

$ systemd-run --user --scope -p MemoryMax=50M sh -c 'cat /sys/fs/cgroup$(cut -d: -f3 /proc/self/cgroup)/memory.max'
52428800

When a cgroup hits memory.max, the kernel's OOM killer kills a process in it with SIGKILL: exit code 137 (128 + 9). docker inspect then shows "OOMKilled": true, and journalctl -k has the kill.

ulimits: limits per process

ulimits (resource limits) cap a single process, not a group. The one that bites in production is open files: every file, socket and connection uses one file descriptor.

$ ulimit -n                       # open-files limit for this shell
1048576
$ ulimit -a                       # every limit
$ grep -E "processes|open files" /proc/$(pgrep -o nginx)/limits   # a running process's limits
Max processes             15109                15109                processes
Max open files            1048576              1048576              files

Each limit has a soft value (what applies) and a hard value (the ceiling a normal user may raise the soft one to). Where to raise them:

"Too many open files" under load means this limit, not the disk. Note that limits.conf doesn't apply to systemd services: they only read their unit file.

Container runtimes: who does what

docker run passes through several programs, each with one job. Knowing them helps when one breaks, and it's what Kubernetes uses too:

docker (CLI)  →  dockerd           images, networks, volumes, the API
              →  containerd        pulls images, runs and watches containers
              →  runc              creates namespaces and cgroups, starts the process, exits
              →  your process      PID 1 inside the container

• An image is a stack of read-only filesystem layers plus settings (command, environment). Running it adds a thin writable layer on top, using overlayfs: delete the container and that layer goes too. Data you keep belongs in a volume.
• OCI (the Open Container Initiative) standardises images and runtimes, so an image built with Docker runs on Podman, containerd or Kubernetes unchanged.
• Podman runs the same images without a daemon, and rootless by default; podman accepts the same commands as docker.

Docker on a Linux server

$ sudo apt install docker.io                      # or Docker's own repository (Packages lesson)
$ sudo usermod -aG docker deploy                  # let deploy use docker (log in again)
$ docker run -d --name web -p 8080:80 --memory 256m --cpus 1 --restart unless-stopped nginx:1.27
$ docker ps                                       # running containers
$ docker logs -f web                              # its stdout and stderr
$ docker exec -it web sh                          # a shell inside its namespaces
$ docker stats --no-stream                        # live CPU and memory per container
$ docker inspect web --format '{{.State.Pid}} {{.State.OOMKilled}}'

From the host, the container is just a process. Its PID from docker inspect works with every tool in this course: ps -fp <pid>, cat /proc/<pid>/limits, sudo nsenter -t <pid> -n ss -tlnp (its network, your tools).

$ systemd-run --user --scope -q -p MemoryMax=50M \
    unshare -r -u -p -f --mount-proc sh -c '
      hostname box1
      echo "hostname: $(hostname)"
      echo "memory:   $(cat /sys/fs/cgroup$(cut -d: -f3 /proc/self/cgroup)/memory.max) bytes"
      ps -eo pid,comm
    '
hostname: box1
memory:   52428800 bytes
    PID COMMAND
      1 sh
      6 ps