The cache miss that costs you minutes
You change a single line in main.go. You run docker build. The terminal spins for forty-five seconds. You did not touch go.mod. You did not add a dependency. The build should take two seconds. Instead, Docker downloads every module again. The cache missed. This happens because Docker and Go track changes differently. Docker looks at file contents. Go looks at cryptographic hashes. When they disagree, you pay for the download every time.
Why Docker and Go disagree on caching
Docker builds images by stacking layers. Each layer is a snapshot of the filesystem. Docker computes a hash of the instruction and the files it touches. If the hash matches a previous build, Docker reuses the layer. If the hash changes, Docker discards the layer and everything above it.
Go manages dependencies using a module cache. It stores downloaded packages in a directory, keyed by the exact hash of go.mod and go.sum. Go checks these files to determine if dependencies have changed. If the hashes match, Go skips the network request and uses the local cache.
The conflict arises when you copy your source code before downloading dependencies. Docker sees the source code change and invalidates the layer that downloads modules. Go is ready to use the cache, but Docker never reaches that step. You must align the build order so Docker reuses the download layer even when source code changes.
Think of Docker layers like a stack of transparent sheets. If you draw a new line on the bottom sheet, you have to redraw every sheet above it. Go's module cache is like a strict librarian. The librarian only hands you a book if the request card matches the catalog exactly. Change the request card, and the librarian assumes you want a different edition.
Docker caches instructions. Go caches hashes. Align them or pay for the download every time.
The naive approach and why it breaks
Here is the common mistake. Copy everything first.
FROM golang:1.23
WORKDIR /app
# Copies all files, invalidating the layer on any change
COPY . .
# Runs every time because the previous layer changed
RUN go mod download
# Compiles the binary
RUN go build -o main .
Docker reads this top to bottom. The COPY . . instruction hashes the entire directory. Change a comment in main.go, and the hash changes. Docker marks that layer as new. Every layer after it rebuilds. go mod download runs again. The Go toolchain checks go.mod and go.sum. If they match the cache, it skips network calls. But Docker already forced a full rebuild. The module cache sits in /go/pkg/mod. It survives inside the builder container, but Docker never reuses that layer. You pay for the download every single time.
The compiler will not stop you. Docker will not warn you. The build just takes longer than it should. You can verify the cache miss by running docker build with the --progress=plain flag. You will see the RUN go mod download step execute on every single push, even when you only changed a comment.
Docker layers are immutable. Go hashes are strict. Separate the concerns or accept the slowdown.
The multi-stage solution
The fix isolates the manifest files. Copy them first. Download. Then copy source.
FROM golang:1.23 AS builder
WORKDIR /app
# Copies only the manifest files to preserve cache when source changes
COPY go.mod go.sum ./
# Downloads modules; this layer stays cached if manifests are unchanged
RUN go mod download
# Copies application source after dependencies are resolved
COPY . .
# Builds a static binary for Linux
RUN CGO_ENABLED=0 GOOS=linux go build -o main .
FROM alpine:latest
# Extracts only the binary from the builder stage
COPY --from=builder /app/main /main
CMD ["/main"]
Docker caches the first RUN instruction because go.mod and go.sum rarely change. When you push a new feature, COPY . . changes, but the dependency download layer stays intact. The build finishes in seconds. The final image contains only the binary and the minimal alpine base. The module cache never leaves the builder stage.
The CGO_ENABLED=0 flag tells the Go compiler to skip C bindings. This produces a fully static binary. Static binaries run anywhere without system libraries. The GOOS=linux flag ensures the compiler targets the container environment even if you build on macOS or Windows. The COPY --from=builder syntax is the standard way to extract artifacts from a previous stage. It keeps the final image clean.
Run go mod tidy before committing. It removes unused dependencies and updates go.sum. A clean manifest means fewer cache invalidations. The community expects the manifest to be the source of truth. A dirty manifest causes unnecessary rebuilds and confuses the cache.
Trust the toolchain. Let the manifest drive the cache.
Common traps and compiler errors
Developers usually run into three problems when adapting this pattern. The first is forgetting go.sum. The second is mixing CGO with alpine. The third is ignoring the module proxy.
If you copy go.mod but leave go.sum behind, the build fails. The compiler rejects the step with go: errors parsing go.mod: go.sum is missing. The go.sum file contains cryptographic hashes for every transitive dependency. Go refuses to download modules without it. Always copy both files together.
If you enable CGO and use alpine, the container crashes at runtime. You get error while loading shared libraries: libc.musl-x86_64.so.1: cannot open shared object file. Alpine uses musl libc. Go expects glibc by default. Disable CGO or switch to a distroless base. Distroless images contain glibc and no shell. They are safer for production workloads.
If your network blocks the default Go module proxy, downloads hang. The compiler prints go: downloading example.com/pkg v1.2.3: Get "https://proxy.golang.org/example.com/pkg/@v/v1.2.3.zip": dial tcp: lookup proxy.golang.org: no such host. Set GOPROXY=https://your-proxy.example.com,direct in the Dockerfile or pass it as a build argument. The module proxy is not optional in modern Go. It provides version resolution and caching. Configure it correctly or your builds will stall.
The scratch image is empty. It contains no shell, no libraries, no package manager. If your binary is static, this is the gold standard. You cannot exec into a scratch container to debug. You must rely on logs. This forces better observability. The community convention for production images is scratch or distroless. alpine is fine for development. scratch contains zero packages. It only runs the binary.
Never commit vendor directories. Use the proxy. Let Go handle resolution.
Adding build metadata without breaking the cache
Teams often want to embed commit hashes, build timestamps, or version strings into the binary. The standard approach uses -ldflags. If you bake the flag directly into the RUN instruction, the layer changes every time you build. Docker invalidates the cache. You lose the dependency download cache again.
The fix is to pass the flags as build arguments. Docker caches the RUN instruction itself. It only rebuilds the layer when the argument values change. This keeps the dependency layer intact while still allowing dynamic metadata.
FROM golang:1.23 AS builder
WORKDIR /app
COPY go.mod go.sum ./
RUN go mod download
COPY . .
ARG VERSION=dev
ARG COMMIT=unknown
# Passes dynamic flags without invalidating the download layer
RUN CGO_ENABLED=0 GOOS=linux go build -ldflags "-X main.version=${VERSION} -X main.commit=${COMMIT}" -o main .
FROM alpine:latest
COPY --from=builder /app/main /main
CMD ["/main"]
Docker treats ARG values as part of the layer hash. If you run docker build --build-arg VERSION=1.0.2 --build-arg COMMIT=abc123, the compile layer updates. The download layer stays cached. The final image gets the correct metadata. This pattern scales to CI pipelines where the version comes from a git tag or a release workflow.
The CMD instruction sets the default command. You can override it at runtime. Use CMD for the main executable. This allows the container orchestrator to pass arguments if needed. The ENTRYPOINT instruction is for scripts that must always run. For a simple binary, CMD is sufficient.
Build arguments isolate change. Cache layers survive. Keep them separate.
When to pick which strategy
Use a single-stage build when you are prototyping locally and do not care about image size or build time. Use a multi-stage build with alpine when you need a small image and your code relies on CGO for system calls. Use a multi-stage build with distroless or scratch when you want the smallest possible footprint and your binary is fully static. Use a dedicated cache volume in CI when you run builds outside Docker and want to persist the module directory across pipeline runs.
Pick the base that matches your threat model. Strip everything else.