Docker Bake¶

Table of Contents¶

• 1. Introduction
• What is Docker Docker Bake?
• What is Docker BuildKit?
• Key Features
• Comparison: Docker Bake vs Docker Compose
• Docker Bake syntax
• Common Commands
• Advanced Features
• Multi-Stage Builds with Matrix
• Build Secrets
• Cache Configuration / Build Caching
• Cache Configuration / Build Caching Workflow
• Target Inheritance

1. Introduction¶

What is Docker Docker Bake?¶

• 👍 Docker Bake is a feature of BuildKit / Docker Buildx that allows you to define and orchestrate complex builds using a high-level build tool that uses to build Docker images defined in configuration files.

• 👍 Docker Bake is written in a declarative language (HCL, JSON, or YAML).

• 👍 Docker Bake is designed for building multiple images or targets concurrently, making it ideal for monorepos, microservices, or any project with several Dockerfiles.

• 👍 Docker Bake extends beyond single Dockerfiles to support complex build scenarios with multiple targets, platforms, and configurations.

• 👍 Docker Bake is designed for CI/CD pipelines, multi-platform builds, and complex build workflows.

What is Docker BuildKit?¶

• 👍 BuildKit is a modern build subsystem for Docker that improves performance, storage management, and caching

• 👍 BuildKit is the engine behind Docker Bake, enabling advanced features like multi-platform builds, caching, and build secrets

• 👍 BuildKit is the default build engine for Docker, replacing the traditional docker build mechanism.

• 👍 BuildKit allows for parallel builds, advanced caching, and better resource utilization.

• 👍 BuildKit supports advanced features like build secrets, SSH forwarding, and more.

• 👍 BuildKit is a key component of the new Docker Engine 20.10 release, which includes the buildkit subcommand and the docker buildx command.

Key Features¶

Comparison: Docker Bake vs Docker Compose¶

[!WARNING] Docker Bake requires Docker Engine v20.10.0 or higher.
Docker Bake requires BuildKit for advanced build features.

Docker Bake syntax¶

[!IMPORTANT] docker-bake.hcl is a declarative build configuration file format.

• Here’s a basic docker-bake.hcl file structure:
• It uses the HashiCorp Configuration Language (HCL) syntax for defining build configurations.
• This example defines two targets: frontend and backend-api, each with its own context, Dockerfile, tags, platforms, and build arguments.

# Define variables
variable "TAG" {
  default = "latest"
}

# Define a user variable for tagging images
# This can be your Docker Hub username or any identifier
variable "USER" {
  default = "username"
}

# Define a group of targets for default builds
group "default" {
  targets = ["frontend", "backend-api"]
}

# Frontend service
target "frontend" {
  # Directory containing the frontend code
  context  = "./frontend"           

  # Use a specific Dockerfile for production
  dockerfile = "Dockerfile.prod" 

  # Tag the image with user and tag         
  tags     = ["${USER}/frontend:${TAG}"]    

  # Build for multiple architectures
  platforms  = ["linux/amd64", "linux/arm64"]   

  # Pass build-time variables to the Dockerfile 
  args = {
    # Set Node.js environment to production
    NODE_ENV = "production" 

    # Set API URL for the frontend
    API_URL = "https://api.codewizard.com"
  }
}

# Node.js backend service
target "backend-api" {
  context  = "./backend" 
  dockerfile = "Dockerfile"
  tags     = ["${USER}/backend:${TAG}"]
  platforms  = ["linux/amd64", "linux/arm64"]
   args = {
    NODE_ENV = "production"
  }
}

• Here is another example of a Docker Bake configuration file: in JSON

{
  "variable": {
    "TAG": {
      "default": "latest"
    },
    "USER": {
      "default": "username"
    }
  },
  "group": {
    "default": {
      "targets": ["frontend", "backend-api"]
    }
  },
  "target": {
    "frontend": {
      "context": "./frontend",
      "dockerfile": "Dockerfile.prod",
      "tags": ["${USER}/frontend:${TAG}"],
      "platforms": ["linux/amd64", "linux/arm64"],
      "args": {
        "NODE_ENV": "production",
        "API_URL": "https://api.codewizard.com"
      }
    },
    "backend-api": {
      "context": "./backend",
      "dockerfile": "Dockerfile",
      "tags": ["${USER}/backend:${TAG}"],
      "platforms": ["linux/amd64", "linux/arm64"],
      "args": {
        "NODE_ENV": "production"
      }
    }
  }
}

Common Commands¶

Advanced Features¶

Multi-Stage Builds with Matrix¶

• What is a Build Matrix?
• A build matrix allows you to create multiple image variations from a single configuration.
• This is useful when you need to build multiple versions of your application or service.
• For example, you might need to build different versions of your application for different environments (e.g., development, staging, production).
• By using a build matrix, you can specify multiple combinations of values for each configuration option.

• Instead of writing separate targets for each combination, Docker Bake automatically generates all possible combinations.

• Example: Building multiple Node.js versions for different environments

target "app" {
  matrix = {
    version = ["18", "20"]
    env = ["dev", "prod"]
  }

  name = "app-node${version}-${env}"
  context = "."
  dockerfile = "Dockerfile"
  tags = ["myapp:node${version}-${env}"]

  args = {
    NODE_VERSION = "${version}"
    NODE_ENV     = "${env}"
  }
}

• Quiz: What will be the output of the above example?

Build Secrets¶

• Build secrets allow you to securely pass sensitive information (like API keys, passwords, or certificates) to your Docker build process without including them in the final image or build history.
• This is particularly useful for keeping sensitive data out of your Docker images and ensuring that it is only available during the build process.
• Build secrets can be stored in a secure location, such as a secure vault or a secret management service, and then referenced during the build process.
• Build secrets are mounted into the build environment at runtime, allowing you to use them without exposing them in the final image.
• Build secrets are not stored in the final image layers, ensuring that sensitive information is not leaked.
• Build secrets are only available during the build process and are not accessible to the final image or container.
• Build secrets are defined in the docker-bake.hcl file using the secret field.
• You can use the --secret flag to pass secrets to the build process.

• Secrets can also be defined in the Dockerfile using the RUN --mount=type=secret syntax.

• Example: Using a secret file during build

target "secure-app" {
  context = "."
  dockerfile = "Dockerfile"
  secret = [
    "id=mysecret,src=./secret.txt"
  ]
  tags = ["myapp:secure"]
}

• Dockerfile:

FROM node:18

# Use the secret during build (it wont be in the final image)
# as mentioned and explained above

RUN --mount=type=secret,id=mysecret \
    API_KEY=$(cat /run/secrets/mysecret) && \
    npm install --registry=https://registry.example.com

Cache Configuration / Build Caching¶

• Docker Bake supports advanced caching strategies to speed up builds and reduce resource usage.
• Caching allows you to reuse previously built layers, which can significantly speed up the build process
• Docker Bake can use different cache sources, such as local files, remote registries, or GitHub Actions cache.
• Caching can be configured using the cache-from and cache-to fields in the docker-bake.hcl file.
• Caching can also be configured using the --cache-from and --cache-to flags when running the docker buildx bake command.
• Caching can be used to speed up builds by reusing previously built layers, which can significantly reduce build times.
• Caching can also be used to reduce resource usage by avoiding unnecessary rebuilds of unchanged layers.
• Caching can be used to share build layers between different builds, which can further speed up the build process.
• Caching can be used to optimize the build process by reusing previously built layers, which can significantly reduce build times.
• Caching can reduce build times from minutes to seconds for unchanged code.
• Example: Using GitHub Actions cache

target "cached-app" {
  context = "."
  dockerfile = "Dockerfile"
  cache-from = ["type=gha"]
  cache-to = ["type=gha,mode=max"]
  tags = ["myapp:cached"]
}

Cache Types:

Cache Configuration / Build Caching Workflow¶

mermaid flowchart TD A[First build] -->|Takes full time, saves cache| B[Second build] B -->|Reuses cached layers, builds only changed parts| C[Result] C -->|Significantly faster build times| D[End]

Target Inheritance¶

• Docker Bake allows you to create a base configuration and reuse it across multiple targets.
• This follows the DRY (Don’t Repeat Yourself) principle - define common settings once and inherit them.

Example: Sharing common settings across multiple builds

# Base target with common settings
target "base" {
  context = "."
  dockerfile = "Dockerfile"
  platforms = ["linux/amd64", "linux/arm64"]
  tags = ["myapp:latest"]
  cache-from = ["type=gha"]
  cache-to = ["type=gha,mode=max"]
}

# Example: Using inheritance to define common settings (1)
target "web" {
  inherits = ["base"]
  tags = ["myapp/web:latest"]
  target = "web"
}

# Example: Using inheritance to define common settings (2)
target "api" {
  inherits = ["base"]
  tags = ["myapp/api:latest"]
  target = "api"
}

How it works:

mermaid flowchart TD A[Base target] -->|Defines common settings | B[Child targets] B -->|Inherit from base and add their specific settings| C[Result] C -->|Both web and api targets automatically get the same platforms and build context| D[End]

Benefits:

• Consistency: Sharing - ensures consistent build configurations
• Reusability: All targets use the same base configuration
• Maintainability: Supports multiple targets with the same settings platforms in one place, affects all children
• Cleaner code: No repetitive configuration
• DRY principle: Avoids duplication of common settings
• Flexibility: Each target can still have its own specific settings
• Scalability: Easily add new targets with shared settings