The invisible request
A request hits your API gateway. It calls the auth service. Auth calls the database. The database times out. Your logs show three separate failure messages across different machines. You cannot tell which call caused the delay, and you cannot reproduce the exact sequence that broke. You need a map of the request's journey. Distributed tracing builds that map.
What tracing actually measures
Tracing tracks a single request as it moves through multiple services. Logging records discrete events. Metrics aggregate numbers over time. Tracing connects the dots. Think of a package moving through a fulfillment center. Each scanning station records a timestamp and a location. The complete sequence forms a trace. Each station's scan is a span. Jaeger renders the timeline. OpenTelemetry provides the scanning hardware.
The OpenTelemetry Go SDK handles the heavy lifting. You configure a tracer provider, attach an exporter that speaks to Jaeger, and wrap your operations in spans. Spans record start times, end times, attributes, and errors. When a span ends, the SDK batches the data and ships it over the network. Jaeger receives the payloads, reconstructs the parent-child relationships, and displays the waterfall diagram you see in the UI.
The minimal setup
Here is the simplest way to wire a Go program to Jaeger. The code initializes the exporter, creates a tracer provider, and registers it globally.
package main
import (
"context"
"fmt"
"log"
"go.opentelemetry.io/otel"
"go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp"
"go.opentelemetry.io/otel/sdk/resource"
"go.opentelemetry.io/otel/sdk/trace"
semconv "go.opentelemetry.io/otel/semconv/v1.26.0"
)
func initTracer() (*trace.TracerProvider, error) {
// OTLP HTTP exporter ships spans to Jaeger's collector endpoint
exporter, err := otlptracehttp.New(context.Background())
if err != nil {
return nil, err
}
// Resource tags identify this service in the Jaeger UI
res := resource.NewWithAttributes(
semconv.SchemaURL,
semconv.ServiceName("order-service"),
)
// Batch spans to reduce network overhead and improve throughput
tp := trace.NewTracerProvider(
trace.WithBatcher(exporter),
trace.WithResource(res),
)
otel.SetTracerProvider(tp)
return tp, nil
}
func main() {
tp, err := initTracer()
if err != nil {
log.Fatal(err)
}
// Ensure pending spans flush before the process exits
defer tp.Shutdown(context.Background())
tracer := otel.Tracer("order-service")
ctx := context.Background()
_, span := tracer.Start(ctx, "startup")
defer span.End()
fmt.Println("Tracer is live")
}
The exporter uses OTLP, the standard protocol that Jaeger accepts. The batch processor collects spans in memory and sends them in chunks. This prevents your application from blocking on every single network call. The resource configuration attaches metadata like the service name, which Jaeger uses to group traces. Calling otel.SetTracerProvider once at startup makes the provider available to every package that imports go.opentelemetry.io/otel. The community convention is to set it early, usually in main, and never change it at runtime.
Wiring it into real code
Spans live inside context.Context. That is the carrier that moves trace data across function boundaries and service boundaries. When you start a span, the SDK returns a new context with the span attached. Pass that context to every downstream call. The downstream code can read the trace ID and span ID, or create child spans that automatically link to the parent.
Here is how a typical HTTP handler looks with tracing attached.
package main
import (
"context"
"net/http"
"go.opentelemetry.io/otel"
"go.opentelemetry.io/otel/attribute"
"go.opentelemetry.io/otel/codes"
)
func handleOrder(w http.ResponseWriter, r *http.Request) {
tracer := otel.Tracer("order-service")
// Attach a new span to the request context
ctx, span := tracer.Start(r.Context(), "HandleOrder")
defer span.End()
// Record request metadata for filtering in Jaeger
span.SetAttributes(
attribute.String("http.method", r.Method),
attribute.String("http.url", r.URL.Path),
)
orderID := "ORD-9921"
_, childSpan := tracer.Start(ctx, "ValidateInventory")
// Simulate downstream work
if orderID == "" {
childSpan.SetStatus(codes.Error, "missing order id")
childSpan.RecordError(fmt.Errorf("invalid order"))
}
childSpan.End()
w.WriteHeader(http.StatusOK)
}
The defer span.End() pattern is standard. It guarantees the span closes even if a panic occurs or an early return triggers. The SDK calculates the duration automatically. Attributes are key-value pairs that Jaeger indexes. You can filter traces by http.url or order.id later. Recording an error marks the span with a red flag in the UI and attaches the error message. The context flows through every function call, carrying the trace state without explicit parameters.
Convention aside: context.Context always goes as the first parameter in Go functions. Name it ctx. Functions that accept a context should respect cancellation and deadlines. Tracing relies on this contract. If you drop the context, you break the trace chain.
Where things break
Tracing adds network calls and memory allocation. Misconfiguration turns it into a performance sink or a data leak.
Forgetting defer span.End() leaves the span open. The batch processor keeps it in memory until the buffer fills or the process exits. You will see incomplete traces and rising memory usage. The compiler will not catch this. You must rely on code review or static analysis.
Passing the wrong context breaks propagation. If you call tracer.Start(context.Background(), "child") instead of tracer.Start(ctx, "child"), the child span loses its parent. Jaeger shows two disconnected traces instead of a tree. The compiler accepts the code because context.Background() is a valid type. The bug only appears in the UI.
Missing tp.Shutdown(context.Background()) drops pending spans. The batch processor buffers data to reduce network calls. If the process terminates before the flush timer fires, those spans vanish. The runtime prints nothing. You simply lose the last few seconds of tracing data. Always defer shutdown in main.
Using a synchronous exporter without batching blocks the calling goroutine until Jaeger acknowledges the payload. If Jaeger is down, your application hangs. The runtime eventually panics with context deadline exceeded if you attach a timeout, or it blocks indefinitely if you do not. The batch processor avoids this by moving network I/O to a background goroutine.
The compiler rejects unused imports with imported and not used. It also complains with cannot use ctx (variable of type context.Context) as string value in argument if you accidentally pass the context where a string is expected. These are straightforward fixes. The harder bugs are logical: missing propagation, unbounded buffers, and unhandled exporter errors.
When to reach for tracing
Tracing is powerful but not universal. Pick the right observability tool for the job.
Use Jaeger with OpenTelemetry when you need to visualize request paths across multiple services and identify latency bottlenecks. Use Prometheus when you need time-series metrics for alerting and capacity planning. Use slog when you need human-readable debug output for developers reading logs in real time. Use plain timing logs when the system is a single process and distributed context propagation adds unnecessary complexity.
Tracing answers "where did this request go and how long did each step take." Metrics answer "is the system healthy right now." Logs answer "what exactly happened at this moment." Combine them for full coverage. Do not replace logging with tracing. Do not replace metrics with tracing. They measure different dimensions of the same system.
Where to go next
- How to Implement Health Checks for Microservices in Go
- How to Use slog (Structured Logging) in Go 1.21+
- How to Use pprof for Production Profiling in Go
Tracing turns invisible request flows into visible maps. Set up the provider once. Propagate context everywhere. Flush before exit.