The universal adapter for data streams
You are building a utility to compress data. The input might come from a file on disk, a live HTTP request body, or a string held in memory. You could write three separate functions: CompressFile, CompressHTTP, and CompressString. That works until you add a database blob or a network socket, and suddenly you need a fourth function.
Go avoids this duplication with interfaces that describe behavior rather than types. The io package defines Reader, Writer, and Closer to standardize how programs consume, produce, and clean up data. Any type that implements these interfaces works with the same functions. You write the logic once, and it runs against files, networks, strings, and custom streams without change.
This design follows the Go convention of implicit interface implementation. You do not declare that a struct implements an interface. The compiler checks the methods. If the struct has the required methods, it satisfies the interface. This keeps packages decoupled and allows third-party types to plug into standard library functions without modification.
Reader, Writer, and the hose analogy
Think of data as water flowing through a hose. A Reader is a tap. You attach your hose to the tap and pull water. You don't need to know if the tap is connected to a kitchen faucet, a fire hydrant, or a water tower. You just call Read, and bytes arrive.
A Writer is a drain. You attach your hose and pour water in. The drain doesn't care where the water came from. It just accepts bytes via Write.
A Closer is the valve. Some resources hold external state, like a file descriptor or a network socket. When you are done, you must turn the valve to release the resource. The Close method handles this cleanup.
The interfaces are small by design. io.Reader has one method. io.Writer has one method. io.Closer has one method. Small interfaces are easy to implement and easy to compose. Go code frequently combines them. io.ReadCloser embeds both Reader and Closer. io.WriteCloser embeds Writer and Closer. This composition lets you pass a stream that can be read and closed, or written and closed, using a single type.
Interfaces describe what you can do, not what you are.
A minimal copy function
The power of these interfaces shows up when you write functions that accept them. Here is a function that copies data from any reader to any writer.
package main
import (
"fmt"
"io"
"strings"
)
// CopyBytes copies data from any Reader to any Writer.
// It returns the number of bytes copied and any error encountered.
func CopyBytes(r io.Reader, w io.Writer) (int64, error) {
buf := make([]byte, 32)
var total int64
for {
// Read fills the buffer, but may return fewer bytes than requested.
n, err := r.Read(buf)
if n > 0 {
// Write only the bytes actually read.
// Write may also return fewer bytes than requested,
// so a robust implementation would loop on Write too.
w.Write(buf[:n])
total += int64(n)
}
// io.EOF signals the end of the stream.
// It is not a failure; it means there is no more data.
if err != nil {
return total, err
}
}
}
func main() {
// strings.NewReader returns a Reader that reads from a string.
src := strings.NewReader("Go makes I/O uniform")
// strings.Builder implements Writer.
var dst strings.Builder
n, err := CopyBytes(src, &dst)
if err != nil && err != io.EOF {
panic(err)
}
fmt.Printf("Copied %d bytes: %s\n", n, dst.String())
}
CopyBytes does not know about strings. It calls Read on the reader and Write on the writer. The strings.Reader fills the buffer with characters from the string. The strings.Builder appends those characters to its internal buffer. The loop continues until Read returns io.EOF.
The code uses buf[:n] when writing. Read can return fewer bytes than the buffer size. This happens often with network streams or when the source runs out of data. Slicing to n ensures you only write the valid bytes. Writing the whole buffer would include stale data from previous iterations.
Real-world usage: HTTP uploads
In production code, you rarely write the copy loop yourself. The standard library provides io.Copy, which handles buffering, partial reads, partial writes, and error checking efficiently. You use io.Copy to move data between streams.
Here is an HTTP handler that saves an uploaded file.
package main
import (
"fmt"
"io"
"net/http"
"os"
)
// SaveRequestBody reads the request body and saves it to a file.
func SaveRequestBody(w http.ResponseWriter, r *http.Request) {
// r.Body implements io.ReadCloser.
// Always close the request body to release the connection.
defer r.Body.Close()
f, err := os.Create("upload.txt")
if err != nil {
http.Error(w, "Cannot create file", http.StatusInternalServerError)
return
}
// defer ensures the file is closed even if io.Copy fails.
defer f.Close()
// io.Copy handles buffering and error checking.
// It reads from r.Body and writes to f until EOF or error.
n, err := io.Copy(f, r.Body)
if err != nil {
http.Error(w, "Copy failed", http.StatusInternalServerError)
return
}
fmt.Fprintf(w, "Saved %d bytes", n)
}
The handler uses defer to close both the request body and the file. defer schedules the call to run when the function returns. This guarantees cleanup even if an error occurs later in the function.
The io.Copy call moves data from the network stream to the disk file. io.Copy uses a buffer internally, so it minimizes system calls. It also checks the return values of Read and Write to handle partial operations correctly.
If you need to limit the amount of data copied, use io.CopyN or io.LimitReader. io.LimitReader wraps a reader and returns io.EOF after a specified number of bytes. This prevents reading more data than expected, which is useful for security and resource control.
Close your resources. The garbage collector does not close files.
Pitfalls and compiler errors
The io interfaces are simple, but the runtime behavior has nuances that trip up developers coming from other languages.
Partial reads and writes are normal. Read can return n < len(p). Write can return n < len(p). This is not a bug. It means the operation completed as far as it could at that moment. Network buffers fill up. Disk writes block. Your code must handle partial results. io.Copy handles this for you. If you write your own loop, check n and retry until all bytes are processed.
io.EOF is a sentinel error. It indicates the end of the stream. It is not a failure. Functions that read until the end should check for io.EOF and treat it as success. Functions that expect a specific amount of data should treat io.EOF as an error if fewer bytes were read than required.
Forgetting to close causes leaks. If you open a file or a network connection, you must close it. The compiler cannot detect missing Close calls. The runtime will leak file descriptors or sockets. Use defer immediately after opening a resource.
Type mismatches are compile-time errors. If you pass a string to a function expecting io.Reader, the compiler rejects it.
The compiler complains with
cannot use s (variable of type string) as io.Reader value in argument: string does not implement io.Reader (missing Read method)if you pass a string directly.
You must wrap the string in a reader. strings.NewReader(s) returns a type that implements io.Reader. Similarly, bytes.NewReader(b) works for byte slices.
io.NopCloser bridges the gap. Sometimes you have an io.Reader but a function requires io.ReadCloser. io.NopCloser wraps a reader and adds a Close method that does nothing. This lets you satisfy the interface without changing the underlying reader.
Partial reads are normal. Write loops that handle them.
When to use what
Go provides many types in the io and related packages. Pick the right tool based on your needs.
Use io.Reader when you need to consume bytes from any source without knowing the source type. Use io.Writer when you need to produce bytes to any destination without knowing the destination type. Use io.Closer when a resource holds external state like a file descriptor or network socket that must be released. Use io.ReadCloser when a stream provides data and also requires cleanup after reading. Use io.Copy when you need to move data between a reader and a writer with optimal buffering and error handling. Use io.NopCloser when you have a reader but need to satisfy a ReadCloser interface. Use io.LimitReader when you must restrict the number of bytes read from a stream. Use bufio.Reader or bufio.Writer when you need to reduce system calls by batching small reads or writes. Use plain sequential code when you don't need concurrency: the simplest thing that works is usually the right thing.
Compose interfaces for flexibility. Use io.Copy for performance.