Copy-on-Write

A base image is eight megabytes. Run fifty containers from it and you do not want fifty copies — and you certainly do not want one container's rm -rf / to damage the image the other forty-nine are using.

The answer is copy-on-write: every container gets what looks like its own private copy, but blocks are only duplicated at the moment something writes to them. Creating one is instant and costs nothing until it is used.

Set up a btrfs filesystem

btrfs has snapshots built in. You don't need a spare disk — a file will do:

sudo apt install -y btrfs-progs

# A sparse 5G file, formatted as a btrfs filesystem
truncate -s 5G ./btrfs-disk.img
mkfs.btrfs -f ./btrfs-disk.img

mkdir -p ./btrfs-mount
sudo mount -o loop ./btrfs-disk.img ./btrfs-mount

truncate -s 5G creates a sparse file: it reports as 5 GB but occupies almost no disk until written to. mount -o loop presents that file to the kernel as a block device, which is what lets a filesystem live inside a regular file.

Five gigabytes according to ls, under four megabytes on disk according to du. That gap is the sparse file.

A subvolume for the base image

A btrfs subvolume is an independently snapshottable tree inside the filesystem. Make one and unpack Alpine into it:

sudo btrfs subvolume create ./btrfs-mount/base-image

curl -o alpine.tar.gz \
  https://dl-cdn.alpinelinux.org/alpine/v3.19/releases/x86_64/alpine-minirootfs-3.19.1-x86_64.tar.gz

sudo tar -xf alpine.tar.gz -C ./btrfs-mount/base-image

This is the immutable thing. Nothing will ever write to it again.

Snapshot it per container

CONTAINER_ID="my-container"

sudo btrfs subvolume snapshot \
  ./btrfs-mount/base-image \
  ./btrfs-mount/$CONTAINER_ID

That completes instantly and consumes essentially no space. The new subvolume references exactly the same blocks as the base image; only when the container modifies a file does btrfs allocate a fresh block for the changed data and repoint that one file. The base image is never touched.

Written in the snapshot, absent from the base. That is the whole guarantee, and it is what makes it safe to hand fifty containers the "same" filesystem.

This is what an image layer is

Docker's layered images are the same idea, generalised. Each instruction in a Dockerfile produces a layer holding only what changed; a running container adds one final writable layer on top and every write lands there. docker commit freezes that writable layer into a new read-only one.

Which explains several behaviours that otherwise look arbitrary:

• Deleting a file doesn't shrink the image. The upper layer records a deletion marker; the original bytes are still in the layer below. RUN apt-get install ... && rm -rf /var/lib/apt/lists/* in one instruction works precisely because the removal happens before the layer is sealed.
• Reordering a Dockerfile changes build time enormously. A layer is cached until something above it changes, so COPY package.json before COPY . . keeps the npm install layer valid across source edits.
• Container writes vanish on removal. The writable layer is deleted with the container. Volumes exist to sit outside this stack entirely.

Tearing it down

Snapshots are subvolumes, so they are removed with a btrfs command rather than rm:

sudo btrfs subvolume delete ./btrfs-mount/$CONTAINER_ID

Next up: networking — giving the container its own interface, its own IP, and a wire back to the host.