Docker BuildKit: Complete Configuration Guide

Docker BuildKit: Complete Configuration Guide

What is Docker BuildKit?

BuildKit is a next-generation build engine shipped with Docker, designed to replace the legacy builder. It introduces a highly parallelized, cache-efficient, and extensible architecture for building container images. Originally released as an experimental feature, BuildKit became the default builder in Docker 23.0 and later, and is fully integrated into the Docker CLI and Docker Compose.

Under the hood, BuildKit uses a concurrent execution model with a directed acyclic graph (DAG) solver. It can parallelize independent build steps, skip unnecessary stages when cache is available, and support advanced features like secrets management, SSH agent forwarding, and multi-stage build caching across different environments. The engine itself can run either as a built‑in component of the Docker daemon or as a standalone buildkitd service, enabling remote builds and integration with CI/CD systems.

Why BuildKit Matters

Moving from the classic builder to BuildKit brings substantial benefits:

• Dramatically faster builds through automatic parallel execution of non‑dependent instructions.
• Smarter layer caching – BuildKit caches intermediate layers and can reuse them even when the Dockerfile context changes, using content‑addressable storage.
• Secure handling of secrets – Secrets can be mounted at build time without persisting in image layers or leaking into the final artifact.
• SSH agent forwarding – Allows the build to access private repositories without embedding credentials in the image.
• Better multi‑stage builds – Automatically skips unused stages and transfers only the necessary artifacts between them.
• Extended syntax – Supports features like RUN --mount=..., COPY --chown, heredocs, and more, enabling cleaner and more expressive Dockerfiles.
• Remote and distributed builds – BuildKit can run as a separate daemon, allowing a CI/CD pipeline to share a remote builder or distribute workloads across worker nodes.

Enabling and Configuring BuildKit

BuildKit is now the default builder in modern Docker installations. You can verify which builder is active and control its configuration via environment variables, daemon‑level settings, and CLI flags. The following sections cover all common configuration scenarios.

1. Enable BuildKit via Environment Variable

The simplest way to force BuildKit for a specific session or script is to set the DOCKER_BUILDKIT environment variable. This works with the docker build command and overrides any default.

# Enable BuildKit for the current shell session
export DOCKER_BUILDKIT=1

# Use it in a single command
DOCKER_BUILDKIT=1 docker build -t myapp:latest .

# Disable it (legacy builder) by setting to 0
DOCKER_BUILDKIT=0 docker build -t myapp:legacy .

2. Daemon‑level Configuration (dockerd)

For persistent, system‑wide BuildKit activation, edit the Docker daemon configuration file, usually located at /etc/docker/daemon.json. Add the "buildkit" key to the "features" section (or "builder" in older versions). After modifying, restart the Docker daemon.

{
  "features": {
    "buildkit": true
  }
}

On newer Docker versions (23+), BuildKit is the default, so this entry may be redundant, but it ensures the feature is explicitly enabled. You can also disable it by setting "buildkit": false.

3. Using the Docker Buildx Client (Recommended)

Docker Buildx is a CLI plugin that exposes advanced BuildKit capabilities, such as creating multi‑platform images, exporting cache, and managing builder instances. Buildx is bundled with Docker Desktop and most package installations. To use it, simply invoke docker buildx build instead of docker build. It automatically uses BuildKit under the hood and provides richer options.

# Create a new BuildKit‑backed builder instance
docker buildx create --name mybuilder --use

# Inspect and verify the builder
docker buildx inspect --bootstrap

# Build with Buildx, targeting multiple platforms
docker buildx build --platform linux/amd64,linux/arm64 -t myapp:multi --push .

4. Standalone BuildKit Daemon (buildkitd)

For environments where you want to decouple the build engine from the Docker runtime (e.g., a CI cluster or a dedicated build server), you can run the BuildKit daemon as a standalone service. This requires the buildkitd binary, usually available from the moby/buildkit repository.

# Run buildkitd with default options (listens on unix socket and TCP)
buildkitd --addr unix:///run/buildkit/buildkitd.sock --addr tcp://0.0.0.0:1234

Then configure the Buildx client to connect to this remote builder:

docker buildx create --name remote-builder --driver remote \
  --addr tcp://buildkit-host:1234 --use

BuildKit Configuration Options

BuildKit can be tuned via a TOML‑based configuration file when running the standalone buildkitd daemon. The config file is typically located at /etc/buildkit/buildkitd.toml or passed via the --config flag. Docker’s integrated BuildKit (through dockerd) respects a subset of these options through the daemon.json, but for advanced tuning the standalone daemon is preferred.

Core Configuration Sections

A typical buildkitd.toml looks like this:

# /etc/buildkit/buildkitd.toml
debug = false

[worker]
  [worker.oci]
    enabled = true
    # maximum number of parallel build steps
    max-parallelism = 4

  [worker.containerd]
    enabled = false

[registry]
  [registry."docker.io"]
    mirrors = ["mirror.company.com"]
    plainHTTP = false

[dns]
  # DNS servers used during build
  nameservers = ["8.8.8.8", "1.1.1.1"]
  options = ["edns0"]

Key areas you can control:

• Worker parallelism – max-parallelism limits the number of concurrent build steps. Adjust based on CPU cores and available memory.
• OCI vs containerd worker – The OCI worker is the default; containerd integration allows BuildKit to share the same containerd runtime as dockerd, reducing overhead.
• Registry mirrors and insecure registries – Specify mirrors for public registries, and allow plain HTTP for internal registries.
• DNS configuration – Define custom DNS servers for the build environment, useful for internal service discovery.

Advanced Dockerfile Features Enabled by BuildKit

With BuildKit, you unlock a powerful set of Dockerfile instructions. These features are not available in the legacy builder. Below are some of the most impactful ones.

Secret Mounts (RUN --mount=type=secret)

Mount secrets such as private keys, tokens, or environment‑specific files without baking them into the final image. Secrets are available only during the execution of the current RUN instruction.

# Dockerfile
FROM alpine

# Secret file mounted at /run/secrets/mykey
RUN --mount=type=secret,id=mysecret,dst=/run/secrets/mykey \
    cat /run/secrets/mykey && \
    # do something with the secret, then it disappears

Supply the secret at build time:

# Using a file as secret
docker build --secret id=mysecret,src=./secret.txt -t myapp .

# Using environment variable via heredoc
docker build --secret id=mysecret,src=/dev/stdin -t myapp <<< "$MY_TOKEN"

SSH Agent Forwarding (RUN --mount=type=ssh)

Forward your SSH agent socket into the build, allowing git clone from private repositories or other SSH‑based operations without embedding keys.

# Dockerfile
FROM golang:1.21

# Use SSH to clone private Go modules
RUN --mount=type=ssh \
    mkdir -p /root/.ssh && \
    ssh-keyscan github.com >> /root/.ssh/known_hosts && \
    go build -o /app ./...

Build with SSH agent forwarding:

docker build --ssh default -t myapp .

Cache Mounts (RUN --mount=type=cache)

Persist directories across builds (e.g., package manager caches) to dramatically speed up subsequent runs. The cache is stored outside the image layers and shared between builds.

# Dockerfile
FROM node:18

# Cache npm dependencies across builds
RUN --mount=type=cache,target=/root/.npm \
    npm install && \
    npm run build

Heredoc Syntax for Multi‑line Commands

BuildKit supports heredocs, letting you write complex scripts directly in RUN or COPY instructions without a separate script file.

# Dockerfile (heredoc example)
FROM python:3.11

RUN <