Caching Docker Layers in CI/CD to Stop Waiting Forever

The four-minute build that should take forty seconds

Your Docker image build is fast locally — forty seconds, most of it served from cache. In CI, the same build takes four minutes every run, regardless of what changed. The CI runner starts fresh, has no layer cache, and rebuilds everything from scratch: downloads the base image, reinstalls all packages, compiles the application. For a source-only change, this is entirely avoidable.

Docker BuildKit supports exporting and importing layer caches to and from external storage. Configure it once and CI gets cache hits for layers that didn't change, bringing build time down to seconds for the common case.

Why local caching doesn't help CI

Local Docker builds store layer cache on disk in Docker's storage backend. On your laptop, layers built Monday are cached for Tuesday's build. CI runners (GitHub Actions, GitLab CI, CircleCI) are typically ephemeral — each job starts in a fresh virtual machine or container with no prior state. Monday's cache is gone.

Some CI systems offer persistent runners (self-hosted GitHub Actions runners, GitLab runner with a persistent host). For these, the local cache persists between runs and no explicit caching configuration is needed. For ephemeral runners — which most teams use — you need to explicitly export the cache somewhere the next run can import it from.

BuildKit supports several cache backends: inline (stored in the image), registry (stored in a registry), S3, Azure Blob, and GitHub Actions cache. Registry-based caching is the most versatile and works across all CI systems that have registry access.

Registry-based caching: the setup

The docker/build-push-action for GitHub Actions handles registry cache export/import natively:

- uses: docker/setup-buildx-action@v3

- uses: docker/login-action@v3
  with:
    registry: ghcr.io
    username: ${{ github.actor }}
    password: ${{ secrets.GITHUB_TOKEN }}

- uses: docker/build-push-action@v5
  with:
    context: .
    push: ${{ github.event_name != 'pull_request' }}
    tags: ghcr.io/${{ github.repository }}:${{ github.sha }}
    cache-from: type=registry,ref=ghcr.io/${{ github.repository }}:buildcache
    cache-to: type=registry,ref=ghcr.io/${{ github.repository }}:buildcache,mode=max

cache-from pulls the cache image before building. cache-to exports the new cache after building. mode=max exports all intermediate layer caches — this is what allows rebuilding from an early cache hit even when a later layer changed. Without mode=max, only the final image's layers are cached, which is equivalent to mode=min.

The buildcache tag is a separate image tag used only for caching. It's not a runnable image — it's a manifest containing BuildKit's internal cache metadata.

GitHub Actions cache: alternative for GHA users

GitHub provides a dedicated cache service via the Actions cache API. BuildKit integrates with it:

- uses: docker/build-push-action@v5
  with:
    cache-from: type=gha
    cache-to: type=gha,mode=max
    push: true
    tags: your-image:${{ github.sha }}

Advantages over registry caching:

• No registry writes for the cache itself — potentially faster for small layer sets
• Native GitHub integration — no authentication setup for the cache store
• Free within the GitHub Actions cache storage limit (10GB per repository)

Disadvantages:

• 10GB limit per repository — large images or many branches can exhaust it
• Cache entries expire after 7 days of no access
• Only available on GitHub Actions

For most projects, type=gha is the fastest path to working CI caching. For larger registries or cross-CI caching, use type=registry.

Multi-stage builds and cache scope

With multi-stage Dockerfiles, each stage can be cached independently. By default, mode=max exports all stage caches. To build the maximum hit rate, ensure your stage structure matches the pattern: slow-changing stages first, fast-changing stages last.

FROM maven:3.9 AS deps           # invalidated only when pom.xml changes
WORKDIR /app
COPY pom.xml .
RUN mvn dependency:go-offline

FROM deps AS build                # invalidated when source changes
COPY src ./src
RUN mvn package -DskipTests

FROM eclipse-temurin:17-jre-alpine  # rarely invalidated
COPY --from=build /app/target/app.jar app.jar
ENTRYPOINT ["java", "-jar", "app.jar"]

In CI, for a source-only change:

• deps stage: cache hit (pom.xml unchanged)
• build stage: rebuilt (source changed)
• Final stage: rebuilt (artifact changed)

Total build time: time to rebuild the build and final stages only. The deps stage (which includes the slow mvn dependency:go-offline) is served from registry cache.

Branch-specific cache isolation

By default, all branches share the same cache key. A large branch with major dependency changes can pollute the cache for the main branch. Use branch-scoped caches with a fallback:

- uses: docker/build-push-action@v5
  with:
    cache-from: |
      type=registry,ref=ghcr.io/${{ github.repository }}:cache-${{ github.ref_name }}
      type=registry,ref=ghcr.io/${{ github.repository }}:cache-main
    cache-to: type=registry,ref=ghcr.io/${{ github.repository }}:cache-${{ github.ref_name }},mode=max

cache-from can take multiple sources. BuildKit tries them in order — if the branch-specific cache has the layer, use it; if not, fall back to the main branch cache. cache-to writes only to the branch-specific tag, preventing branch builds from overwriting the main cache.

Measuring cache effectiveness

Add timing to your pipeline to measure the improvement:

- name: Build image
  id: build
  uses: docker/build-push-action@v5
  with:
    # ... cache config ...

- name: Report build time
  run: echo "Build completed at $(date). Check the build step's duration above."

GitHub Actions shows each step's duration in the workflow run view. Compare before and after adding cache configuration. For a typical Spring Boot or Node.js service:

• Cold build (no cache): 3–6 minutes
• Warm build (source-only change, deps cached): 30–90 seconds
• Full cache hit (nothing changed): 5–15 seconds

The improvement is most dramatic on projects with heavy dependency installation steps. If your build was already fast (under 60 seconds), caching offers less benefit.

The self-hosted runner shortcut

If you use self-hosted GitHub Actions runners on persistent machines, you don't need registry caching at all — the Docker layer cache persists on the runner's disk between jobs. The same is true for GitLab runners configured with pull_policy: if-not-present on a persistent host.

For teams with budget for persistent CI infrastructure, this is simpler to operate than registry cache management. For teams using ephemeral cloud runners, registry caching is the only option.

What to configure today

If your CI pipeline does docker build without any cache configuration, add the two cache-from and cache-to lines with type=gha for GitHub Actions, or type=registry for any other CI system. This is a one-PR change that meaningfully reduces CI build time on every subsequent run.