How does Docker’s layered filesystem affect image build performance and storage efficiency?

15 Eyl 10:26

Docker’s layer model is essentially a copy‑on‑write stack, much like how a VM snapshot works, but far lighter weight. Each `RUN`, `COPY` or `ADD` creates a read‑only diff that the next step builds on. Because the build engine hashes the command and its inputs, it can reuse any previously built diff — exactly the same way a VM snapshot can be cloned without re‑executing the whole boot sequence. In practice this means an incremental build that only changes the last few commands finishes in seconds, while the unchanged layers are just linked in. If you compare that to a monolithic Dockerfile (or a single‑layer tarball) where any change forces the whole archive to be rebuilt, the performance gain is dramatic. The trade‑off shows up in storage: every layer lives as a separate directory under `/var/lib/docker/overlay2` (or the driver you use). If you frequently rebuild with small tweaks, you’ll accumulate many almost‑identical layers until you prune them (`docker image prune` or `docker system prune`). By contrast, a VM image stores a single disk file, so you only have one copy, but you lose the fine‑grained cache and the build time spikes for every change. A practical strategy is to keep the most volatile parts (e.g., `pip install`, `npm install`) in the later layers and group stable base packages early. Also, consider using BuildKit’s inline cache (`--metadata-file`) or external cache mounts so you can share layers across projects, which reduces both build time and the total disk footprint. In short, Docker’s layered FS gives you build‑time speed at the cost of a few extra megabytes per layer; proper layer ordering and regular pruning keep that overhead lean.