How to Handle Rate Limiting in HTTP Clients in Go

Go does not have a built-in rate limiter in the standard net/http client, but you can enforce rate limits by wrapping the http.Client with a golang.org/x/time/rate limiter. Create a rate.Limiter with your desired tokens per second and use a custom Transport that waits for tokens before making requests.

import (
	"net/http"
	"time"
	"golang.org/x/time/rate"
)

func NewRateLimitedClient(maxRequestsPerSecond float64) *http.Client {
	limiter := rate.NewLimiter(rate.Limit(maxRequestsPerSecond), 1)
	return &http.Client{
		Transport: &rateLimitTransport{
			limiter: limiter,
			base:    http.DefaultTransport,
		},
	}
}

type rateLimitTransport struct {
	limiter *rate.Limiter
	base    http.RoundTripper
}

func (t *rateLimitTransport) RoundTrip(req *http.Request) (*http.Response, error) {
	if err := t.limiter.Wait(req.Context()); err != nil {
		return nil, err
	}
	return t.base.RoundTrip(req)
}

Alternatively, set the GODEBUG environment variable to http2client=0 if you need to disable HTTP/2 multiplexing to simplify connection-based rate limiting, though this is rarely required for standard rate limiting.

Rate limiting prevents your Go program from sending too many requests to a server at once, which could get your IP blocked. You achieve this by adding a "traffic cop" (a limiter) to your HTTP client that pauses requests if you've sent too many recently. This ensures your app behaves politely and stays within the server's allowed usage limits.