The universal adapter for byte streams
You're writing a function to parse JSON. First, it reads from a file. Then your boss says, "Can it read from a URL?" You copy the function, change os.Open to http.Get, and duplicate the parsing logic. A week later, you need to read from a database blob. You're about to copy-paste again when you notice a pattern. Every source gives you bytes. Every destination wants bytes. Go solves this with one interface that appears in almost every standard library package: io.Reader.
io.Reader is the Swiss Army knife of Go I/O. It lets you write functions that work with files, network connections, memory buffers, and compressed streams without changing a single line of logic. The interface is tiny, but the impact is massive. It enables composition, testing, and code reuse across the entire ecosystem.
The contract: fill the bucket
io.Reader is a contract. It says, "Give me a bucket, and I'll fill it with bytes until it's full or I run out of data." The interface has a single method:
type Reader interface {
Read(p []byte) (n int, err error)
}
You pass a slice of bytes. The reader writes into that slice. It returns how many bytes it wrote and whether an error occurred.
Think of a water tap. You hold a bucket under the tap. The tap doesn't care what's in the bucket or where the bucket came from. It just pours water until the bucket is full or the water stops. The tap is the io.Reader. The bucket is the byte slice. The amount of water is n. If the pipe bursts or the water runs out, that's the err.
The method signature tells the whole story. p is the buffer. n is the count of bytes written. err is the status. The reader modifies the slice in place. It does not allocate a new slice. It does not return the data in a new variable. You provide the storage; the reader fills it.
Convention aside: receiver names in Go are usually one or two letters matching the type. Use (r *MyReader) for a receiver, not (this *MyReader) or (self *MyReader). The community expects short names. It keeps the code clean and signals that the receiver is just a value, not a hidden object model.
io.Reader is the universal adapter for data streams.
Minimal implementation
Here's the simplest implementation. A reader that emits a fixed string and then stops. This shows the mechanics of tracking state and returning the correct counts.
package main
import (
"fmt"
"io"
)
// StringReader wraps a string and implements io.Reader.
type StringReader struct {
data []byte
pos int
}
// Read fills the provided buffer with data from the string.
func (r *StringReader) Read(p []byte) (n int, err error) {
// Return EOF if we've already sent all the data.
if r.pos >= len(r.data) {
return 0, io.EOF
}
// Copy as much as fits into the buffer.
n = copy(p, r.data[r.pos:])
// Advance the position so the next call continues where we left off.
r.pos += n
return n, nil
}
func main() {
reader := &StringReader{data: []byte("Hello, Go!")}
buf := make([]byte, 100)
n, err := reader.Read(buf)
fmt.Printf("Read %d bytes: %q, err: %v\n", n, buf[:n], err)
}
The copy function is safe. It calculates the smaller of the source and destination lengths and copies that many bytes. It prevents buffer overflows. The reader returns n so the caller knows exactly how much valid data is in the buffer. The buffer might be larger than the data returned. The caller must respect n.
What happens at runtime
When you call Read, the implementation checks its internal state. If there's data, it copies bytes into the slice you provided. It returns the count of bytes copied. The caller uses that count to know how much valid data is in the slice.
The critical detail is partial reads. Read can return fewer bytes than requested. It might return 1 byte. It might return 1000 bytes. It depends on the underlying system. Network sockets often return partial data because the OS fills the buffer with whatever is available right now. Files usually return full data but can be interrupted by signals.
The caller must handle partial reads. If you need exactly 100 bytes, you might need to call Read multiple times. The io package provides helpers like io.ReadFull that loop for you until the buffer is full or an error occurs.
Error handling follows the standard Go pattern. if err != nil { return err } is verbose by design. The community accepts the boilerplate because it makes the unhappy path visible. You cannot accidentally ignore an error.
io.EOF is special. It means "no more data." It's not a failure. It's the end of the stream. Functions like io.ReadAll treat io.EOF as a signal to stop and return the data collected so far. If you write a manual loop, you must check for io.EOF specifically. Other errors indicate problems like broken connections or permission denied.
Always check the error. io.EOF is a signal, not a panic.
Realistic usage: limiting and copying
In real code, you rarely implement io.Reader from scratch. You usually compose existing readers. The io package provides tools like io.LimitReader, io.MultiReader, and io.TeeReader. Here's a realistic scenario: reading a request body in an HTTP handler, but limiting the size to prevent a denial-of-service attack.
package main
import (
"fmt"
"io"
"net/http"
)
// MaxUploadSize limits the request body to 1 megabyte.
const MaxUploadSize = 1 << 20
// HandleUpload reads the request body safely.
func HandleUpload(w http.ResponseWriter, r *http.Request) {
// Wrap the body reader to reject payloads larger than the limit.
limited := io.LimitReader(r.Body, MaxUploadSize+1)
buf := make([]byte, MaxUploadSize+1)
n, err := io.ReadFull(limited, buf)
// Check if the payload exceeded the limit.
if n > MaxUploadSize {
http.Error(w, "Payload too large", http.StatusRequestEntityTooLarge)
return
}
// Process the data if it fits.
fmt.Fprintf(w, "Received %d bytes\n", n)
}
io.LimitReader wraps another reader and stops after a certain number of bytes. It returns io.EOF when the limit is reached, even if the underlying reader has more data. This lets you enforce constraints without writing custom logic.
The killer app for io.Reader is io.Copy. It reads from a source and writes to a destination. It handles buffering internally. It's the standard way to move data between streams.
// CopyFile copies data from src to dst using io.Copy.
func CopyFile(dst io.Writer, src io.Reader) (int64, error) {
// io.Copy handles buffering and partial reads automatically.
return io.Copy(dst, src)
}
io.Copy accepts an io.Writer and an io.Reader. It works with files, network connections, and memory buffers. It returns the number of bytes copied and any error. This function can copy a file to stdout, a request body to a file, or a compressed stream to a decompressed stream. The logic is identical.
Convention aside: "Accept interfaces, return structs" is the most common Go style mantra. Functions should accept io.Reader, not *os.File. This allows callers to pass any reader, including mocks for testing. Return concrete types like *os.File so callers can access specific methods if needed.
Wrap readers to enforce constraints. Don't trust the input size.
Pitfalls and compiler errors
Partial reads are the most common bug. If you call Read once and assume the buffer is full, you'll miss data. The buffer might contain only part of the message. Use io.ReadFull or loop until you have the expected amount.
Another pitfall is ignoring n. The buffer might be partially filled. If you use buf instead of buf[:n], you read garbage or zeros. The slice length is the capacity of the buffer, not the amount of data. Always slice the buffer to n before processing.
io.Reader is not safe for concurrent use by default. If two goroutines call Read on the same reader, you get data races. The internal state gets corrupted. Protect shared readers with a mutex or pass copies. Some readers like strings.Reader are safe for concurrent reads because they don't mutate state. Check the documentation.
If you try to pass a type that doesn't implement Reader, the compiler rejects the program with cannot use x (type string) as io.Reader value in argument. The compiler checks interface satisfaction at compile time. You cannot pass a string directly. You must wrap it with strings.NewReader.
Forget to import io and you get undefined: io from the compiler. Forget to use an import and you get imported and not used. Go is strict about imports.
The worst goroutine bug is the one that never logs. If a goroutine blocks on a read that never completes, it leaks. Always have a cancellation path. Use context.Context to signal when to stop reading.
io.Reader is sequential. Protect shared readers with a mutex or pass copies.
When to use io.Reader
Use io.Reader when you need to abstract the source of byte data. Use io.Writer when you need to abstract the destination of byte data. Use io.ReadWriter when a type supports both reading and writing, like a network connection. Use io.Closer when the resource requires explicit cleanup, like closing a file or connection. Use bufio.Reader when you need to buffer reads for performance or parse lines and tokens. Use bytes.Reader when you have a byte slice and want a reader without allocating a new buffer. Use strings.NewReader when you have a string and want a reader without converting to bytes first. Use io.Pipe when you want to connect a reader and writer goroutine for streaming data between them.
Pick the interface that matches the operation. Compose them for complex behavior.