The container barrier
You run your Go app locally and it works perfectly. You build the Docker image, spin up the container, and suddenly the app crashes with a panic you've never seen, or it hangs silently. The logs show nothing useful. You're staring at a black box. This is the "works on my machine" trap, amplified by containers. Debugging inside Docker feels like trying to fix a watch while wearing oven mitts. You need a way to step through the code, inspect variables, and see exactly where the logic diverges from your expectations.
Go comes with a debugger called dlv (Delve). It's the standard tool for stepping through Go code. When you run dlv, it attaches to your program and lets you pause execution, set breakpoints, and inspect the state. The challenge with Docker is that your code runs inside a container, isolated from your host machine. dlv needs to talk to your IDE or terminal. The solution is to run dlv inside the container in "headless" mode, listening on a TCP port. You map that port to your host, and your IDE connects to it as if the debugger were running locally.
Think of dlv as a mechanic's lift. The container is the garage. You can't reach the car inside the garage, so you install a remote control for the lift. You press a button on your phone (IDE), and the lift inside the garage raises the car so you can see underneath. The TCP port is the signal between your phone and the lift.
Build flags that matter
Go compiles to machine code aggressively. By default, the compiler optimizes for speed and size. It inlines functions, reorders instructions, and eliminates variables it thinks are unused. This makes the binary fast, but it destroys the mapping between your source code and the running program. The debugger relies on DWARF debug information to map machine instructions back to your source lines. If the compiler optimizes too hard, the mapping breaks. Variables might disappear, or breakpoints might land in the wrong place.
To debug effectively, you must disable these optimizations. The flags are -gcflags="all=-N -l". The -N flag disables optimizations. The -l flag disables inlining. The binary gets larger and slower, but the debugger works.
Run gofmt before building. The debugger doesn't care about formatting, but your code review bot will. Most editors run gofmt on save, so this is automatic. Don't argue about indentation; let the tool decide.
Here's a minimal main.go to demonstrate. The code is simple so you can focus on the debugging setup.
package main
import (
"fmt"
"time"
)
// main starts the application and simulates a workload.
func main() {
// Simulate a delay to give you time to attach the debugger if needed.
time.Sleep(2 * time.Second)
// This loop processes items. Set a breakpoint here to inspect 'i'.
for i := 0; i < 5; i++ {
fmt.Printf("Processing item %d\n", i)
}
}
The minimal setup
You need three pieces: a binary built with debug flags, a container image with dlv installed, and an IDE configuration.
The Dockerfile builds the binary with the debug flags and installs dlv. The CMD runs dlv in headless mode. The --headless flag tells dlv not to open a text-based UI. The --listen flag opens a TCP port. The --api-version=2 flag uses the modern protocol. The --accept-multiclient flag allows the IDE to reconnect if it disconnects. Without --accept-multiclient, dlv stops when the IDE disconnects, and you have to restart the container. Always use --accept-multiclient. Restarting containers to reconnect is a waste of time.
# Dockerfile
FROM golang:1.22 AS builder
WORKDIR /app
COPY go.mod go.sum ./
RUN go mod download
COPY . .
# Build with debug flags. -N disables optimizations, -l disables inlining.
# Without these, the debugger cannot map variables to source lines reliably.
RUN go build -gcflags="all=-N -l" -o /debug-app .
FROM golang:1.22
# Install delve debugger.
RUN go install github.com/go-delve/delve/cmd/dlv@latest
WORKDIR /app
COPY --from=builder /debug-app .
# Expose the debug port.
EXPOSE 2345
# Run delve in headless mode.
# --headless: no TUI. --listen: TCP port. --api-version=2: modern protocol.
# --accept-multiclient: allows IDE to reconnect if it disconnects.
CMD ["dlv", "exec", "./debug-app", "--headless", "--listen=:2345", "--api-version=2", "--accept-multiclient"]
The IDE configuration tells your editor to connect to the remote debugger. In VS Code, this is a launch.json entry. The remotePath must match the WORKDIR in the Dockerfile. If they differ, breakpoints won't hit. The IDE maps your local files to the container files using this path.
{
"version": "0.2.0",
"configurations": [
{
"name": "Docker Debug",
"type": "go",
"request": "launch",
"mode": "remote",
"remotePath": "/app",
"port": 2345,
"host": "127.0.0.1",
"showLog": true,
"trace": "verbose"
}
]
}
Build the image and run the container. Map port 2345 to your host.
docker build -t debug-go .
docker run -p 2345:2345 debug-go
Start the debug session in your IDE. Set a breakpoint in main.go. The debugger pauses at the breakpoint. You can inspect i, step over, and continue.
Realistic HTTP server
Real apps aren't just loops. They have HTTP servers, database connections, and environment variables. A common pattern is to debug an HTTP handler.
Convention aside: HTTP handlers receive a context.Context via r.Context(). Pass it to database calls. If the request cancels, the context signals the goroutine to stop. Functions that take a context should respect cancellation and deadlines. context.Context always goes as the first parameter, conventionally named ctx.
Here's a realistic handler. It reads an environment variable and returns a response. This is a common place for bugs. You might forget to set the variable in the container, or you might parse it wrong.
package main
import (
"fmt"
"log"
"net/http"
"os"
"strconv"
)
// handleHealth returns a simple status response.
func handleHealth(w http.ResponseWriter, r *http.Request) {
// Check environment variable. This is a common place for bugs.
port := os.Getenv("PORT")
if port == "" {
port = "8080"
}
// Parse the port to an integer. Handle the error explicitly.
// if err != nil { return err } is verbose by design.
// The community accepts the boilerplate because it makes the unhappy path visible.
portInt, err := strconv.Atoi(port)
if err != nil {
http.Error(w, "Invalid port", http.StatusInternalServerError)
return
}
fmt.Fprintf(w, "OK on port %d", portInt)
}
// main starts the HTTP server.
func main() {
http.HandleFunc("/health", handleHealth)
log.Println("Server starting...")
// log.Fatal stops the program if the server fails to start.
log.Fatal(http.ListenAndServe(":8080", nil))
}
Convention aside: Don't pass a *string. Strings are already cheap to pass by value. Passing a pointer adds indirection without benefit. Pass string directly.
Convention aside: Public names start with a capital letter. Private start lowercase. No keywords like public or private. HandleHealth would be public. handleHealth is private. Keep helpers private unless you need to export them for testing.
Convention aside: Interfaces are accepted, structs are returned. "Accept interfaces, return structs" is the most common Go style mantra. If you mock the HTTP handler for testing, use an interface.
The Dockerfile is similar, but you might want to pass environment variables. Use docker run -e PORT=9090.
# Dockerfile
FROM golang:1.22 AS builder
WORKDIR /app
COPY go.mod go.sum ./
RUN go mod download
COPY . .
# Build with debug flags.
RUN go build -gcflags="all=-N -l" -o /debug-app .
FROM golang:1.22
RUN go install github.com/go-delve/delve/cmd/dlv@latest
WORKDIR /app
COPY --from=builder /debug-app .
EXPOSE 2345
CMD ["dlv", "exec", "./debug-app", "--headless", "--listen=:2345", "--api-version=2", "--accept-multiclient"]
Run the container with the env var.
docker run -p 2345:2345 -e PORT=9090 debug-go
Set a breakpoint in handleHealth. Hit the endpoint with curl localhost:8080/health. The debugger pauses. Inspect port, portInt, and err.
Pitfalls and errors
Debugging in Docker has specific gotchas.
If you forget to install dlv in the image, the container fails to start. The error is exec: "dlv": executable file not found in $PATH. Install dlv in the Dockerfile.
If you use dlv debug instead of dlv exec, dlv tries to build the code inside the container. This can fail if the source code isn't available, or it can duplicate work. Use dlv exec when the binary is already built. Use dlv debug only when you want the debugger to build and run in one step, which is rare in Docker.
If the dlv version in the container doesn't match the Go version, you might get protocol errors. The IDE plugin might complain with could not connect to delve. Ensure dlv is built with the same Go version as the binary, or use a recent version of dlv that supports your Go version.
If you forget -gcflags, the debugger attaches but breakpoints fail to hit. The IDE shows could not find function or variables appear as <optimized out>. The compiler rejects this with no error, but the debugger silently fails. Always use -gcflags="all=-N -l" for debug builds.
If you use CGO_ENABLED=1, the binary links against libc. The container must have libc installed. If you use CGO_ENABLED=0, the binary is static and doesn't need libc. For debugging, CGO_ENABLED=0 is safer because it avoids dependency issues. If you need CGO, ensure the base image has the C library.
Docker's seccomp profile can block ptrace, which dlv uses to debug. On older Docker versions, you might need --security-opt seccomp=unconfined. Modern Docker allows dlv by default. If you get ptrace: operation not permitted, check your seccomp profile.
Convention aside: Goroutine leaks happen when the goroutine waits on a channel that never gets closed. Always have a cancellation path. If you spawn a goroutine in the handler to do background work, ensure it finishes. Otherwise, the container holds resources forever. The worst goroutine bug is the one that never logs.
Decision matrix
Use dlv exec when running inside Docker, where the binary is already built and you just need to attach the debugger to the existing executable.
Use dlv debug when you are developing locally and want the debugger to build and run the code in one step.
Use dlv attach when the process is already running and you need to pause it mid-flight, though this requires ptrace permissions and is harder to configure in containers.
Use log.Printf and structured logging when debugging in production, where attaching a debugger is too slow or risky.
Use pprof when you need to profile CPU or memory usage rather than stepping through logic errors.
Debug symbols make the binary heavy. Strip them for production. Build with -ldflags="-s -w" to remove symbol tables and DWARF info. The binary gets smaller and faster, but you can't debug it. Keep debug builds separate from production builds.