The plumbing problem
You are building a service that receives uploaded files and forwards them to a remote storage backend. The naive approach writes a manual loop: allocate a byte slice, call Read, check the length, call Write, repeat until the stream ends. You handle partial reads, you track byte counts, you propagate errors, and you repeat the same pattern across five different handlers. The code works, but it duplicates logic that the standard library already solved.
Go treats data movement as a universal pattern. Whether you are reading from a disk file, a network socket, a database cursor, or a string in memory, the interface contract stays the same. The io package abstracts the read-write loop into three focused functions. io.Copy moves everything until the source closes. io.CopyN stops after a fixed number of bytes. io.CopyBuffer lets you supply your own memory slice to avoid repeated allocations. They share the same signature shape, but each targets a different streaming requirement.
Streams are just sequences of bytes. Treat them as such.
How the io.Copy family works
Go's I/O model revolves around two interfaces: io.Reader and io.Writer. A reader exposes a Read(p []byte) (n int, err error) method. A writer exposes a Write(p []byte) (n int, err error) method. The io.Copy family bridges them. You pass a destination writer and a source reader, and the function handles the transfer.
Think of it like a conveyor belt between two warehouses. The belt has a fixed capacity. Workers load boxes from the source onto the belt, the belt moves them to the destination, and the process repeats until the source runs out of boxes. You do not need to manage the belt speed, the box placement, or the handoff timing. The function manages the buffer, the loop, and the error propagation. You only provide the endpoints.
The default buffer size is a compromise. Trust it until benchmarks say otherwise.
Minimal example
Here is the simplest way to transfer all data from one stream to another.
package main
import (
"io"
"os"
)
// main demonstrates basic io.Copy usage.
func main() {
// Open a source file for reading.
src, err := os.Open("input.txt")
if err != nil {
panic(err)
}
// Close the source when the function returns.
defer src.Close()
// Open a destination file for writing.
dst, err := os.Create("output.txt")
if err != nil {
panic(err)
}
// Close the destination when done.
defer dst.Close()
// Transfer all bytes from src to dst.
// The function allocates a 32KB buffer internally.
n, err := io.Copy(dst, src)
if err != nil {
panic(err)
}
// n holds the total bytes transferred.
// os.Stdout is an io.Writer, so we can print directly.
os.Stdout.WriteString("Copied " + string(rune(n)) + " bytes\n")
}
The function returns two values: the number of bytes transferred and an error. If the transfer completes successfully, the error is nil. The byte count is useful for logging, progress tracking, or verifying that a file was fully transmitted. In many cases you only care about the error. The Go community convention is to discard the byte count with an underscore when you do not need it. Writing _, err := io.Copy(dst, src) signals that you considered the return value and intentionally dropped it.
Accept interfaces, return structs. The io.Copy family accepts any io.Reader and io.Writer, which is why it works with files, network connections, and in-memory buffers without modification.
What happens under the hood
When you call io.Copy, the function checks whether the destination writer implements io.ReaderFrom. If it does, the function delegates the work to the destination's optimized method. This is how os.File and bytes.Buffer achieve high throughput without copying data through an intermediate slice. The delegation bypasses the generic loop entirely.
If delegation is not available, io.Copy allocates a 32KB byte slice. It enters a tight loop. Each iteration calls src.Read(buf). The read returns the number of bytes filled and an error. If the error is io.EOF, the loop stops and the function returns the accumulated byte count with a nil error. If the error is anything else, the function returns immediately with that error. Between reads, it calls dst.Write(buf[:n]) to push the data forward. It repeats until the source closes or an error occurs.
The 32KB default balances memory pressure with syscall overhead. Reading one byte at a time triggers a system call per byte, which collapses throughput. Reading 1MB at once wastes RAM and increases latency for small transfers. 32KB aligns with typical page sizes and network MTUs. The function also handles short reads gracefully. If Read returns fewer bytes than requested, the function writes exactly what it received and continues. It never assumes the stream will fill the buffer completely.
The default buffer size is a compromise. Trust it until benchmarks say otherwise.
Realistic streaming scenario
Production code rarely copies files on disk. It streams data between network connections, HTTP responses, and compression layers. Here is how the three variants fit into a web server that serves file previews and full downloads.
package main
import (
"io"
"net/http"
"os"
)
// servePreview handles requests for a fixed-size file preview.
func servePreview(w http.ResponseWriter, r *http.Request) {
// Open the source file.
f, err := os.Open("large-archive.tar")
if err != nil {
http.Error(w, "file not found", http.StatusNotFound)
return
}
// Ensure the file closes after the handler returns.
defer f.Close()
// Transfer exactly 1024 bytes to the HTTP response.
// CopyN stops after the limit, even if the file is larger.
n, err := io.CopyN(w, f, 1024)
if err != nil && err != io.EOF {
http.Error(w, "read error", http.StatusInternalServerError)
return
}
// Set the content length header for the client.
w.Header().Set("Content-Length", fmt.Sprintf("%d", n))
}
The io.CopyN variant is useful when you need to enforce a hard limit. It stops after the requested byte count. If the source ends before reaching the limit, it returns io.EOF along with the actual byte count. You must check for io.EOF explicitly if you care about distinguishing a truncated stream from a complete one.
High-throughput servers often reuse buffers to reduce garbage collection pressure. Allocating a 32KB slice per request adds up under load. io.CopyBuffer lets you allocate once and pass the same slice across multiple transfers.
package main
import (
"io"
"net/http"
"os"
)
// serveFull handles full file downloads with a shared buffer.
func serveFull(w http.ResponseWriter, r *http.Request) {
// Open the source file.
f, err := os.Open("large-archive.tar")
if err != nil {
http.Error(w, "file not found", http.StatusNotFound)
return
}
// Close the file when the handler finishes.
defer f.Close()
// Reuse a 64KB buffer across requests.
// Larger buffers reduce syscall frequency for big files.
buf := make([]byte, 64<<10)
// Transfer all data using the pre-allocated buffer.
// The function never reslices or reallocates.
_, err = io.CopyBuffer(w, f, buf)
if err != nil {
http.Error(w, "transfer failed", http.StatusInternalServerError)
return
}
}
Buffer reuse matters when your server handles thousands of concurrent streams. The garbage collector does not need to track and sweep thousands of short-lived slices. The CPU cache stays warm. The memory allocator stays quiet. You trade a small amount of per-request state for predictable throughput.
Pick the right tool for the stream. Don't overengineer the plumbing.
Common pitfalls and compiler feedback
The io.Copy family behaves predictably, but a few patterns trip up developers new to Go's I/O model.
The first pitfall is treating io.EOF as a failure. The function swallows io.EOF and returns nil for the error. This is intentional. Reaching the end of a stream is expected behavior, not an exception. If you need to detect premature closure, check the byte count against the expected size or use io.CopyN and inspect the returned error directly.
The second pitfall is passing the arguments in the wrong order. The function signature is Copy(dst io.Writer, src io.Reader). Beginners often reverse them. The compiler catches this immediately with a type mismatch error like cannot use src (type io.Reader) as io.Writer in argument. The fix is to swap the parameters. The naming convention helps: destination first, source second.
The third pitfall is ignoring context cancellation. The io.Copy family does not accept a context.Context. If your stream runs for minutes, a client disconnect or a timeout will leave the goroutine blocked on a read or write call. The convention is to wrap long-lived streams with a cancellation-aware reader or writer, or to run the copy in a goroutine that listens on a context done channel. Functions that take a context should respect cancellation and deadlines. The context.Context always goes as the first parameter, conventionally named ctx. When you cannot pass context directly, you manage it at the boundary layer.
The fourth pitfall is assuming the buffer is shared safely across goroutines. io.CopyBuffer does not synchronize access to the provided slice. If two goroutines call it with the same buffer simultaneously, you get a data race. Allocate separate buffers per goroutine, or use a worker pool with one buffer per worker.
EOF is not an error. It is the expected end of the line.
When to reach for which function
Use io.Copy when you need to transfer an entire stream without knowing its size in advance. Use io.CopyN when you must enforce a hard byte limit, such as serving file previews or reading fixed-length protocol headers. Use io.CopyBuffer when you run many transfers in a tight loop and want to eliminate per-request buffer allocations. Use a manual read-write loop when you need to transform data on the fly, such as applying encryption, compression, or line-by-line parsing.
The simplest thing that works is usually the right thing.