Image layer caching is a technique used to speed up builds and reduce image size in Docker. It works by reusing previously built layers during the build process, instead of building them from scratch every time. This can significantly reduce the time and resources required for a build.
There are three types of layers that can be contributed to an image: build, cache, and launch.
- Build layers: These are the directory that will be accessible by subsequent buildpacks.
- Cache layers: These are the directory that will be included in the cache.
- Launch layers: These are the directory that will be included in the run image as a single layer.
Docker uses a cache of the built image layers, and any changes in one layer will cause the recreation of all subsequent layers. To specify a tagged image to be used as a cache source for the docker build command, use the --cache-from argument. Multiple images can be specified as a cache source by using multiple --cache-from arguments.
In addition to images, the cache can also be pulled from special cache manifests generated by buildx or the BuildKit CLI (buildctl). These manifests (when built with the type=registry and mode=max) can be used as a cache source for subsequent builds.
Another way to use image layer caching is by using a distributed immutable cache, which is a shared cache in the container registry. This allows an arbitrary number of builders to use the shared cache without breaking the reproducibility of previously assembled layers and images. This approach is based on the ideas of Multi-Version Concurrency Control (MVCC) and optimistic locking.
For example, when using a Node.js application, the werf documentation provides information on how to use a distributed immutable cache for the build process. This can be applied to other programming languages and frameworks as well.
Sources:
- Layer Types ยท Cloud Native Buildpacks
- Web performance: Cache efficiency exercise - Engineering at Meta
- Make Docker-in-Docker builds faster with Docker layer caching | GitLab
- Web Caching Basics: Terminology, HTTP Headers, and Caching Strategies | DigitalOcean
- docker build | Docker Docs
- memcached Image Tags History Chainguard Academy
- Things you need to know | First steps | Node.js | werf
- Things you need to know | First steps | Rails | werf
- Things you need to know | First steps | Spring Boot | werf
- Things you need to know | First steps | Laravel | werf
