The connection pool is the real client
You are building a bot to fetch prices from an e-commerce API. You write a quick script using http.Get, run it, and it works. You deploy it, and after an hour the process crashes with too many open files. You add a timeout, and now requests fail randomly with context deadline exceeded. The server is fine. The network is fine. The issue is how you are using the http.Client.
Go's HTTP client is not a simple function that sends a request and returns. It is a stateful object that manages a pool of connections to servers. The default settings work for low-traffic scripts, but they hide connection pooling details that bite you at scale. If you treat the client as a black box, you will eventually leak connections, hit file descriptor limits, or suffer from poor performance due to excessive TCP handshakes.
How the client manages connections
The http.Client struct holds a Transport field. The transport is the engine that handles the actual network I/O. It maintains a cache of idle connections. When you make a request, the client asks the transport for a connection. If an idle connection to that host exists, the transport reuses it. If not, the transport opens a new TCP connection, negotiates TLS, and sends the request.
When the response body is closed, the connection returns to the pool. If you do not close the body, the connection stays open until the garbage collector runs. The garbage collector does not know about HTTP connections. It only sees memory. The connection leaks, and eventually you run out of file descriptors.
The default client, http.DefaultClient, is a global singleton. It has sensible defaults for most cases, but it has no timeout. A request to a slow server can hang indefinitely. In production code, you create your own client instance with explicit timeouts and reuse it across your application. Creating a new client for every request destroys the connection pool and forces a new handshake every time.
Minimal request
Here is the simplest way to make a request using the default client, which http.Get uses under the hood.
// http.DefaultClient is a global singleton with no timeout.
// It reuses connections automatically via its transport.
client := http.DefaultClient
// Get creates a request and executes it using the client.
// It returns a response and an error.
resp, err := client.Get("https://example.com")
if err != nil {
// The compiler rejects the program if err is unused.
// Go forces you to handle errors explicitly.
log.Fatal(err)
}
defer resp.Body.Close()
// Always close the body to return the connection to the pool.
// Even if you do not read the body, closing it is required.
The http.Get function is a convenience wrapper. It calls http.DefaultClient.Get. Using http.Get directly is fine for throwaway scripts, but it prevents you from configuring timeouts or transport settings. The convention is to use http.Client directly so you can control the behavior.
Walkthrough: what happens when you call Get
When you call Get, the client constructs a *http.Request internally. It passes the request to the transport's RoundTrip method. The transport looks up the host in its connection cache. If it finds an idle connection, it checks the connection's expiration time. If the connection is still valid, the transport writes the request headers and body over the existing socket.
If no idle connection exists, the transport dials a new TCP connection. It performs the TLS handshake if the URL uses HTTPS. It sends the request and waits for the response. The response headers arrive first. The client returns the *http.Response to your code. The response body is an io.ReadCloser. You read from it to get the data.
When you call resp.Body.Close(), the client checks if the connection can be reused. If the server sent Connection: keep-alive and the response body was fully read, the connection goes back to the pool. If the body was not fully read, the connection is closed. Partial reads leave data in the socket buffer, which can corrupt the next request on that connection. The client handles this by closing the connection, but it is better to read the full body or use io.Copy to drain it.
Realistic client with context and POST
In production code, you need timeouts, cancellation, and the ability to send POST requests with bodies. You also use context to control long-running operations. The convention is to pass context.Context as the first argument to functions, usually named ctx. Functions that take a context should respect cancellation and deadlines.
// Create a client with a timeout to prevent hanging.
// The timeout covers the entire request lifecycle including DNS.
client := &http.Client{
Timeout: 5 * time.Second,
}
// Context adds cancellation and deadlines.
// It is the standard way to control long-running operations.
ctx, cancel := context.WithTimeout(context.Background(), 3*time.Second)
defer cancel()
// NewRequestWithContext binds the context to the request.
// The client will respect the context deadline.
req, err := http.NewRequestWithContext(ctx, http.MethodPost, "https://api.example.com/data", strings.NewReader(`{"key":"value"}`))
if err != nil {
log.Fatal(err)
}
req.Header.Set("Content-Type", "application/json")
// Do executes the request and returns the response.
// It handles retries and redirects automatically.
resp, err := client.Do(req)
if err != nil {
log.Fatal(err)
}
defer resp.Body.Close()
// A 404 is not an error in Go's http.Client.
// You must check the status code yourself.
if resp.StatusCode != http.StatusOK {
log.Fatalf("unexpected status: %d", resp.StatusCode)
}
The Timeout field on the client is a hard deadline for the entire request. The context deadline is more flexible. You can cancel the request early based on user action or a parent deadline. The client respects both. If either expires, the request fails with a timeout error.
The if err != nil pattern is verbose by design. The community accepts the boilerplate because it makes the unhappy path visible. You cannot accidentally ignore an error. If you try to assign a string to a *http.Response, the compiler complains with cannot use "text" as *http.Response value in assignment. If you forget to import a package, you get undefined: pkg. If you import a package and do not use it, the compiler rejects the program with imported and not used.
Tuning the transport for performance
The Transport struct controls connection pooling, timeouts, and protocol versions. The default transport has limits that can hurt performance in high-throughput scenarios. The most common gotcha is MaxIdleConnsPerHost. The default is 2. If you are making many requests to the same host, only two connections stay idle. The rest are closed after use, forcing new handshakes.
// Transport configures the underlying connection behavior.
// It controls pooling, timeouts, and protocol versions.
transport := &http.Transport{
MaxIdleConns: 100,
MaxIdleConnsPerHost: 100, // Increase for high-throughput to single host.
IdleConnTimeout: 90 * time.Second,
TLSHandshakeTimeout: 10 * time.Second,
}
client := &http.Client{
Transport: transport,
Timeout: 30 * time.Second,
}
The MaxIdleConns field sets the total number of idle connections across all hosts. The MaxIdleConnsPerHost sets the limit per host. If you are scraping a single site, set MaxIdleConnsPerHost higher. The IdleConnTimeout determines how long an idle connection stays in the pool. If the server closes the connection earlier, the client detects this on the next use and opens a new one.
You can also disable HTTP/2 if you need compatibility with buggy servers. The default transport enables HTTP/2 automatically. You can disable it via the GODEBUG environment variable or by configuring the transport.
# Disable HTTP/2 for the client via environment variable.
# This affects all HTTP clients in the process.
export GODEBUG=http2client=0
Or configure the transport explicitly in code.
// ForceAttemptHTTP2 controls HTTP/2 usage.
// Setting it to false disables HTTP/2 for this client.
transport := &http.Transport{
ForceAttemptHTTP2: false,
}
The ForceAttemptHTTP2 field is a boolean. Setting it to false prevents the transport from upgrading to HTTP/2. This is useful for testing or when interacting with servers that have broken HTTP/2 implementations.
Pitfalls and compiler traps
The http.Client has several traps that catch beginners. The most common is forgetting to close the response body. The compiler does not catch this. It is a runtime leak. If you forget defer resp.Body.Close(), the connection is not returned to the pool. The connection stays open until the garbage collector runs. The garbage collector does not know about HTTP connections. The connection leaks, and you eventually run out of file descriptors.
Another trap is treating HTTP errors as Go errors. A 404 response is not an error in Go's http.Client. It returns a response with StatusCode 404 and err is nil. You must check the status code yourself. This trips up developers coming from Python's requests library, which raises exceptions for non-2xx status codes. In Go, you check resp.StatusCode manually.
If you try to read the body after closing it, you get an error, but the compiler will not stop you. The resp.Body is an io.ReadCloser. Reading from a closed reader returns io.EOF or an error. If you pass a nil body to Do, the compiler complains with net/http: nil Body or similar depending on the version. If you use the wrong method, the compiler rejects the program with cannot use "GET" as http.Method value in argument.
Goroutine leaks happen when the goroutine waits on a channel that never gets closed. Always have a cancellation path. If you spawn a goroutine to read the response body, ensure the goroutine exits when the context is cancelled. The worst goroutine bug is the one that never logs.
The convention is to use _ to discard values intentionally. If you do not need the response body, you can do resp, err := client.Do(req); if err != nil { ... }; resp.Body.Close(); _ = resp.StatusCode. This discards the status code and signals to the reader that you considered it and chose to drop it. Use _ sparingly with errors. Discarding an error hides bugs.
Decision matrix
Use http.Get when you are writing a throwaway script and do not need configuration. Use http.Client with Timeout when you need a simple hard deadline for the entire request. Use http.Client with context when you need to cancel requests based on user action or parent deadlines. Use a custom http.Transport when you need to tune connection pooling, disable HTTP/2, or add custom TLS verification. Use http.NewRequestWithContext when you are sending a POST or PUT request with a body and headers. Use http.DefaultClient when you want the global defaults and do not care about timeouts. Use a RoundTripper wrapper when you need to add middleware like logging or authentication headers. Use io.Copy to drain the response body when you do not need the data but must close the connection properly.
Close the body or lose the connection. Transport is the engine. Client is the dashboard. 404 is a success in Go's eyes. Check the status code.