The Engine Underneath

The docker command is a client. Everything it appears to do is done by a chain of programs behind it, and in production you eventually meet all of them — usually because one has gone wrong and the CLI is reporting it politely.

The chain

docker (CLI)
   │  REST over /var/run/docker.sock
dockerd (the daemon)
   │  gRPC
containerd
   │  one shim per container
containerd-shim-runc-v2
   │  fork/exec
runc  ──►  clone(), setns(), cgroups, pivot_root  ──►  your process

• dockerd owns the Docker-shaped concepts: images with tags, networks, volumes, the build system, the API.
• containerd owns container lifecycle and image distribution. It is a CNCF project used directly by Kubernetes, with Docker as one client among several.
• The shim is one process per container, and it is the piece that matters operationally: it holds the container's stdio and exit status, and it is the reason you can restart dockerd without killing every running container.
• runc does the actual work — namespaces, cgroups, pivot_root, capabilities — then execs your process and exits. It is a short-lived program, not a supervisor.

That last pair is the whole trick from Containers From Scratch: a container is a host process with different namespaces attached. /proc/<pid>/ answers every question about it directly, without going through Docker at all — which is exactly what you want when Docker is the thing misbehaving.

The specifications underneath

Two OCI specifications separate what Docker is from what a container is:

• The image spec — a manifest, a config, and layers as content-addressed tarballs. Why an image built by Docker runs under Podman, containerd, or a Kubernetes node with no Docker on it.
• The runtime spec — the config.json describing namespaces, mounts, cgroup limits, capabilities, and a seccomp profile. runc consumes exactly this.

Reading a real config.json is the fastest way to see everything Docker configures on your behalf, and every field maps onto something in this course.

Where things are on disk

/var/lib/docker/            # images, containers, volumes, build cache
/var/lib/containerd/        # containerd's own state
/etc/docker/daemon.json     # daemon configuration
/var/run/docker.sock        # the API socket

Two rules about /var/lib/docker. Do not modify it by hand — the daemon holds state in memory and will not agree with your edits. And give it its own filesystem on any real host: a full root filesystem takes the whole machine down, where a full Docker filesystem only stops new builds.

Configuring the daemon

/etc/docker/daemon.json, applied on daemon restart:

{
  "log-driver": "local",
  "log-opts": { "max-size": "10m", "max-file": "3" },
  "live-restore": true,
  "default-ulimits": { "nofile": { "Soft": 65536, "Hard": 65536 } },
  "default-address-pools": [
    { "base": "10.201.0.0/16", "size": 24 }
  ]
}

Every one of those is a production lesson someone learned the hard way. Log rotation, because the default is unbounded. live-restore, so containers survive a daemon restart explicitly. File descriptor limits, because the default is low for a busy server. And address pools, because Docker's default 172.17.0.0/16 collides with corporate VPN ranges more often than seems statistically reasonable, and the symptom is one subnet of the office becoming unreachable from containers.

Next up: security — dropping the privileges a container gets by default and does not need.