The cost of knocking on the door
You write a scraper that hits ten endpoints. The first request takes 200ms. The second takes 200ms. The tenth takes 200ms. Total time is 2 seconds. You check the network logs and see a full TCP handshake and TLS negotiation for every single request. You are paying the setup cost every time. The server is sitting there, ready to send data, but your client keeps knocking on the door, waiting for the lock to turn, then leaving immediately after getting the data.
This happens when you create a new http.Client for every request, or when you misconfigure the transport. Go's net/http package handles connection pooling automatically, but only if you share the client and let the transport do its job. Connection reuse eliminates the handshake overhead. It turns a 200ms request into a 10ms request once the connection is warm.
How the transport manages the pool
Think of a connection like a phone line. Dialing takes time. If you call, say "hello", hang up, and call again for the next sentence, you waste time dialing. Connection pooling keeps the line open. You say your sentence, wait a bit, then say the next one without redialing.
Inside every http.Client lives a Transport. The transport is the engine that talks to the network. It maintains a bag of open connections. When you make a request, the transport reaches into the bag, grabs a connection to the right host, and sends the data. When the response comes back, it puts the connection back in the bag instead of throwing it away.
If you create a new http.Client for every request, you get a new bag every time. The bag starts empty. You pay the setup cost. You throw the bag away. Repeat. The fix is simple: create one client, share it everywhere, and close the response body after every request.
Convention aside: HTTP calls are long-lived operations. They should always accept a context.Context as the first argument. The context carries cancellation signals and deadlines. If the user cancels the request, the context tells the transport to drop the connection immediately rather than waiting for a timeout. This keeps your pool healthy and prevents goroutine leaks.
The baseline pattern
Here is the simplest way to reuse connections: one global client, shared across all calls, with bodies closed immediately.
package main
import (
"fmt"
"net/http"
)
// Global client shares a Transport, which holds the connection pool.
// Creating one client and reusing it is the standard pattern.
var client = &http.Client{}
func main() {
// First request establishes a connection.
resp1, err := client.Get("https://httpbin.org/get")
if err != nil {
fmt.Println("Error:", err)
return
}
// Closing the body signals the transport that the connection is free.
// If you skip this, the connection leaks and the pool fills up.
resp1.Body.Close()
// Second request reuses the open connection from the pool.
// No new TCP handshake or TLS negotiation happens here.
resp2, err := client.Get("https://httpbin.org/get")
if err != nil {
fmt.Println("Error:", err)
return
}
// Always close the body to return the connection to the pool.
resp2.Body.Close()
}
When you instantiate &http.Client{}, the Transport field is nil. The client detects this and swaps in http.DefaultTransport. This default transport is a singleton shared by the entire process. It has a pool of up to 100 idle connections total, and up to 2 per host. For a small script, this works fine. For a high-throughput service, two connections per host might be a bottleneck.
The compiler rejects unused variables. If you call client.Get and don't assign the result, you get resp declared and not used. You must capture the response and close the body. The if err != nil check is verbose by design. The community accepts the boilerplate because it makes the unhappy path visible. You cannot accidentally swallow an error in Go.
Share the client. Tune the transport. Close the body.
Tuning for concurrency
When you run many requests at once, the default limits can cause churn. The transport creates connections as needed, but it only keeps a few idle. If you fire 50 concurrent requests to one host, the transport creates 50 connections. When they finish, it keeps only MaxIdleConnsPerHost alive. If that value is 2, it closes 48 connections. The next batch of requests has to open 48 new connections.
Here is a tuned client for concurrent workloads. The MaxIdleConnsPerHost setting matches the expected concurrency level.
package main
import (
"net/http"
)
// Shared client with tuned transport for concurrent access.
// MaxIdleConnsPerHost should match your expected concurrency per host.
var client = &http.Client{
Transport: &http.Transport{
// Total idle connections across all hosts.
MaxIdleConns: 100,
// Idle connections per host.
// If you fire 50 requests to one API, set this to 50 or higher.
MaxIdleConnsPerHost: 10,
},
}
The MaxIdleConns setting is the global cap. MaxIdleConnsPerHost is the per-host cap. The effective limit is the minimum of the two constraints. If MaxIdleConns is 10 and you have 5 hosts, you can have 2 per host. If MaxIdleConnsPerHost is 10 and MaxIdleConns is 10, you can have 10 total, but only 2 per host if you have 5 hosts.
Here is how you use the tuned client with goroutines. The transport is safe for concurrent use. Multiple goroutines can call client.Do simultaneously without locks.
import (
"context"
"fmt"
"net/http"
"sync"
)
// Fetcher uses the shared client for concurrent requests.
// It respects context cancellation to prevent goroutine leaks.
func Fetcher(ctx context.Context, url string) error {
// NewRequestWithContext binds the request to the context.
// This allows cancellation to interrupt the network call.
req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
if err != nil {
return err
}
// Do executes the request using the client's transport.
// The transport selects a connection from the pool.
resp, err := client.Do(req)
if err != nil {
return err
}
// Close the body to return the connection to the pool.
// If you return early without closing, the connection is lost.
defer resp.Body.Close()
// Process response here.
return nil
}
func main() {
var wg sync.WaitGroup
// Launch 20 concurrent requests.
for i := 0; i < 20; i++ {
wg.Add(1)
go func() {
defer wg.Done()
// All goroutines share the client.
// The transport handles the concurrency safely.
Fetcher(context.Background(), "https://httpbin.org/get")
}()
}
wg.Wait()
}
Convention aside: The receiver name is usually one or two letters matching the type. If you define a method on a client wrapper, use (c *Client), not (this *Client) or (self *Client). Go idioms favor brevity in receivers.
HTTP/2 changes the math
HTTP/2 multiplexes streams over a single TCP connection. With HTTP/1.1, you need a pool of connections to handle concurrency. With HTTP/2, one connection can carry many streams. The transport handles this automatically. If the server supports HTTP/2, you get multiplexing for free.
The transport attempts HTTP/2 by default. You can force it with ForceAttemptHTTP2: true, though this is usually redundant. When HTTP/2 is active, MaxIdleConnsPerHost matters less because the transport reuses the single connection for multiple streams. You still need to share the client, but the pool size has less impact on throughput.
If you are talking to a legacy HTTP/1.1 server, the pool is critical. Tune MaxIdleConnsPerHost to match your concurrency. If you are talking to a modern HTTP/2 server, focus on MaxConnsPerHost to limit total connections, and trust the multiplexing.
Draining the body for reuse
If you close resp.Body without reading it, the transport assumes the connection is in a bad state. It discards the connection instead of returning it to the pool. This happens because the server might have sent data that you didn't consume. The next request on that connection would see garbage data.
To reuse the connection, you must read the body fully, or drain it. Draining reads the data and throws it away. This signals to the transport that the connection is clean.
import (
"io"
"net/http"
)
// DrainBody reads the response body and discards it.
// This allows the connection to be reused even if you don't need the data.
func DrainBody(resp *http.Response) error {
// Copy to io.Discard reads the stream without storing it.
// The underscore discards the byte count, which is usually irrelevant.
_, err := io.Copy(io.Discard, resp.Body)
if err != nil {
return err
}
// Close the body after draining.
// This returns the connection to the pool.
return resp.Body.Close()
}
Convention aside: The underscore _ discards a value intentionally. _, err := io.Copy(...) says "I considered the second return value and chose to drop it". Use it sparingly with errors, but freely with counts or indices you don't need.
Timeouts and stale connections
Connections sit idle in the pool. Servers close idle connections to save resources. If the client thinks a connection is open but the server closed it, the request fails. The transport detects this and retries, but it adds latency.
IdleConnTimeout controls how long the client keeps an idle connection. The default is 90 seconds. If your requests are bursty with long gaps, you might want a shorter timeout to avoid stale connections. If you have steady traffic, a longer timeout keeps connections warm.
ResponseHeaderTimeout limits the time waiting for the server's headers. TLSHandshakeTimeout limits the time for the TLS negotiation. Set these based on your latency requirements. If a server takes 10 seconds to respond, you probably want to drop the connection and try another.
If you forget to set a timeout, the client waits indefinitely. You might see context deadline exceeded errors if the caller has a context, or the program hangs silently. Always set timeouts on the transport or pass a context with a deadline.
A leaked connection is a silent resource drain. Close the body or pay the price.
Pitfalls and compiler errors
Creating a new http.Client inside a loop or handler is a common mistake. Each client gets a fresh transport. The transport starts with an empty pool. You pay the TCP/TLS cost every time. Performance tanks. The compiler won't stop you. You have to watch for the pattern.
Setting DisableKeepAlives to true forces a new connection for every request. Use this only if you are debugging a specific server that breaks keep-alive, or if you are behind a proxy that requires it. Otherwise, leave it false.
If you ignore the response, the compiler rejects the program with resp declared and not used. You must capture the response. If you capture it but forget to close the body, the connection leaks. The pool fills up. Eventually, the client blocks waiting for a free connection. You might see context deadline exceeded errors, or the program hangs.
The worst goroutine bug is the one that never logs. A connection leak doesn't panic. It just slows down until the service dies. Use defer resp.Body.Close() to ensure the connection returns to the pool even if you return early.
Decision matrix
Use a shared http.Client with default transport for simple scripts or low-throughput services.
Use a shared http.Client with a custom Transport when you need to tune connection limits for high concurrency.
Use DisableKeepAlives only when a legacy server or proxy breaks persistent connections.
Use a per-request client only in unit tests where isolation matters more than speed.
Use http.NewRequestWithContext when you need to cancel requests or enforce deadlines.
Connection pooling is automatic. You just have to share the client.