Docker Multi-Stage Build Optimization: Reducing Image Size by 80%

Introduction

Docker images can grow surprisingly large if you are not careful. My Go API server image was over 1.2GB, and my Python ML pipeline image was nearly 3GB. These sizes were causing problems: slow CI/CD pipeline runs, excessive storage consumption on the Docker registry, and long deployment times as nodes pulled large images.

The solution was multi-stage builds combined with careful optimization of the build process. By the end of this work, the Go image was reduced to 28MB and the Python image to 450MB — an 80-85% reduction. Here is exactly what I did.

Environment

Docker version: 24.0.7
Go application: API server, ~50MB compiled binary
Python application: ML pipeline with numpy, pandas, scikit-learn
CI/CD: GitHub Actions
Registry: Docker Hub

Problem

The initial Dockerfiles were simple and functional:

# Dockerfile.go-api (before optimization)
FROM golang:1.21

WORKDIR /app
COPY . .
RUN go build -o server ./cmd/server

EXPOSE 8080
CMD ["./server"]

# Dockerfile.ml-pipeline (before optimization)
FROM python:3.11

WORKDIR /app
COPY requirements.txt .
RUN pip install -r requirements.txt
COPY . .

EXPOSE 8000
CMD ["python", "main.py"]

Image sizes after building:

docker images
# REPOSITORY         TAG       SIZE
# go-api             latest    1.21GB
# ml-pipeline        latest    2.87GB

The Go image included the entire Go toolchain, source code, and build artifacts. The Python image included pip, build dependencies, and cached package files. Neither image needed any of these in production.

Analysis

I profiled the image layers to identify the largest contributors:

docker history go-api:latest
# IMAGE          CREATED BY                                      SIZE
# abc123         CMD ["./server"]                                0B
# def456         COPY . .                                        15MB
# ghi789         RUN go build -o server ./cmd/server             650MB (build cache)
# jkl012         WORKDIR /app                                    0B
# mno345         /bin/sh -c apt-get update && apt-get install   200MB
# pqr678         /bin/sh -c #(nop) FROM golang:1.21             800MB

The Go base image alone was 800MB. The build cache from go build added another 650MB. The final binary was only 50MB, but the build artifacts were included in the final image.

For the Python image:

docker history ml-pipeline:latest
# IMAGE          CREATED BY                                      SIZE
# abc123         CMD ["python", "main.py"]                       0B
# def456         COPY . .                                        20MB
# ghi789         pip install -r requirements.txt                 1.8GB
# jkl012         COPY requirements.txt .                         1KB
# mno345         /bin/sh -c apt-get update && apt-get install   400MB
# pqr678         /bin/sh -c #(nop) FROM python:3.11             900MB

The Python base image was 900MB, pip install added 1.8GB, and apt packages added 400MB.

Solution

I rewrote both Dockerfiles using multi-stage builds:

Go API server:

# Stage 1: Build
FROM golang:1.21-alpine AS builder

RUN apk add --no-cache git ca-certificates tzdata

WORKDIR /app
COPY go.mod go.sum ./
RUN go mod download

COPY . .
RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build \
    -ldflags="-w -s -X main.version=$(git describe --tags --always)" \
    -o server ./cmd/server

# Stage 2: Final image
FROM scratch

COPY --from=builder /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/
COPY --from=builder /usr/share/zoneinfo /usr/share/zoneinfo
COPY --from=builder /app/server /server

EXPOSE 8080
ENTRYPOINT ["/server"]

Key optimizations:

golang:1.21-alpine — Smaller base image (250MB vs 800MB)
Separate go mod download — Dependencies cached in a separate layer
CGO_ENABLED=0 — Static binary, no libc dependency
-ldflags="-w -s" — Strip debug info and symbol table
FROM scratch — Final image has only the binary and certs

Python ML pipeline:

# Stage 1: Build dependencies
FROM python:3.11-slim AS builder

RUN apt-get update && apt-get install -y --no-install-recommends \
    build-essential \
    libpq-dev \
    && rm -rf /var/lib/apt/lists/*

WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir --prefix=/install -r requirements.txt

# Stage 2: Final image
FROM python:3.11-slim

RUN apt-get update && apt-get install -y --no-install-recommends \
    libpq5 \
    curl \
    && rm -rf /var/lib/apt/lists/*

COPY --from=builder /install /usr/local
WORKDIR /app
COPY . .

EXPOSE 8000
CMD ["python", "main.py"]

Key optimizations:

python:3.11-slim — Slim base (150MB vs 900MB)
--no-cache-dir — Do not cache pip packages
--prefix=/install — Install to a separate directory for clean copying
--no-install-recommends — Skip recommended packages
Remove apt lists with rm -rf /var/lib/apt/lists/*

I also created a .dockerignore file to exclude unnecessary files from the build context:

# .dockerignore
.git
.github
*.md
LICENSE
docker-compose*.yml
.env*
node_modules
__pycache__
*.pyc
.pytest_cache
.coverage
htmlcov
dist
build
*.egg-info
test
tests
vendor

After rebuilding:

docker images
# REPOSITORY         TAG       SIZE
# go-api             latest    28MB
# ml-pipeline        latest    450MB

For the CI/CD pipeline, I also added BuildKit caching:

# .github/workflows/docker.yml
jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v3

      - name: Build and push
        uses: docker/build-push-action@v5
        with:
          context: .
          push: true
          tags: myregistry/go-api:latest
          cache-from: type=gha
          cache-to: type=gha,mode=max

BuildKit with GitHub Actions cache reduced the build time from 4 minutes to 45 seconds on subsequent builds.

Lessons Learned

Multi-stage builds are essential — Build tools, source code, and build artifacts should never be in the final image. Use a build stage and a minimal runtime stage.
Use slim or alpine base images — The difference between python:3.11 (900MB) and python:3.11-slim (150MB) is enormous. Always prefer slim variants.
Cache dependencies separately — Copy and install dependencies before copying application code. Docker caches layers, so dependency installation only reruns when requirements.txt or go.mod changes.
Use .dockerignore — Excluding .git, node_modules, and test files reduces the build context size and speeds up the build.
Enable BuildKit caching — BuildKit with layer caching can reduce rebuild times by 80% in CI/CD pipelines.

This blog does not accept any external sponsorships, affiliate marketing, or ad revenue.