Building mutable data without the allocation tax
You need to assemble a large string from many small pieces. Maybe you're formatting a log entry with timestamps, levels, and messages. Or you're reading a network stream where the total size isn't known until the end. In Go, strings are immutable. Every time you concatenate two strings, the runtime allocates a new block of memory and copies the data. Do that in a loop, and your program spends more time moving memory than doing work. bytes.Buffer solves this. It's a mutable sequence of bytes that grows efficiently as you add data.
The buffer as a reusable notepad
Think of a bytes.Buffer like a reusable notepad. You write on it, erase it, and write again. The notepad expands automatically if you need more space. Unlike a string, which is like a printed label you can't change, the buffer lets you modify the content in place. Under the hood, it's a slice of bytes with a read index and a write index. It manages the memory allocation for you, doubling the capacity when needed to keep writes fast.
The buffer implements both io.Reader and io.Writer. This means you can pass it to any function that expects a reader or a writer. It fits seamlessly into Go's I/O ecosystem. You can copy data from a file into a buffer, or from a buffer into a network connection, without writing custom loops.
Minimal example
Here's the basic pattern: create a buffer, write some data, read it back, and reset it for reuse.
package main
import (
"bytes"
"fmt"
)
func main() {
// Buffer starts empty. Capacity is zero until the first write.
var buf bytes.Buffer
// WriteString avoids the []byte conversion overhead for string literals.
buf.WriteString("Hello, ")
// Write accepts any io.Writer-compatible data, like a byte slice.
buf.Write([]byte("World!"))
// String() copies the buffer content into a new string.
// Use this when you need a Go string type.
fmt.Println(buf.String())
// Bytes() returns a slice pointing to the buffer's internal storage.
// No copy happens here, but the slice is valid only until the next write.
data := buf.Bytes()
fmt.Println(string(data))
// Reset clears the buffer and reuses the underlying memory.
// This avoids allocation if you plan to fill it again.
buf.Reset()
}
Buffers grow. Strings don't.
Inside the buffer: slices, indices, and growth
A bytes.Buffer holds a slice of bytes internally. It tracks two positions: where the next read happens and where the next write happens. When you call Write, data goes into the slice at the write position. If the slice is full, the buffer allocates a larger slice, copies the existing data, and continues. This amortized growth keeps writes efficient.
WriteString is a specialized version that takes a string directly. It skips the conversion to []byte that Write would require, saving a tiny bit of overhead. Use WriteString for string literals and Write for byte slices or when you're passing data through an interface.
Read pulls data from the read position. It's useful when you treat the buffer as an io.Reader. ReadString reads until a delimiter, like a newline. It returns the data read so far, even if the delimiter isn't found. This is handy for parsing line-based protocols.
Bytes returns a slice of the current content. It points to the internal memory, so it's fast. However, if you write more data later, that slice might become invalid. String creates a copy. Use this when you need a stable string that won't change if the buffer mutates.
Reset sets the read and write positions back to zero. It doesn't free the memory. The buffer keeps the allocated slice so you can reuse it without reallocating. This is the idiomatic way to clear a buffer. Don't reassign buf = bytes.Buffer{} if you want to reuse capacity.
Reset reuses memory. Reassignment wastes it.
Len, Cap, and pre-allocation
buf.Len() returns the number of bytes available to read. It's the difference between the write index and the read index. buf.Cap() returns the capacity of the underlying slice. This is the total space allocated, regardless of how much data is stored.
When the buffer grows, the capacity usually doubles. You can use buf.Grow(n) to pre-allocate space if you know the approximate size. This reduces reallocations. If you're building a large message, call Grow once at the start. The buffer will allocate enough space in one go.
// Grow pre-allocates space.
// Useful when you know the approximate size to avoid multiple reallocations.
buf.Grow(1024)
// WriteString adds data to the buffer.
// Since capacity is pre-allocated, no reallocation happens here.
buf.WriteString("Pre-allocated content goes here efficiently.")
Pre-allocate when you know the size. Let the buffer grow when you don't.
Realistic usage: buffering a stream
You often encounter streams where you need to inspect data before processing it. Maybe you need to check the size, validate a header, or buffer the entire payload for a request. bytes.Buffer works well with io.Copy to read from any io.Reader.
io.Copy detects if the destination implements io.WriterTo or the source implements io.ReaderFrom. bytes.Buffer implements both. This allows io.Copy to use optimized paths that avoid intermediate buffers. The data moves directly between the source and the buffer's internal slice.
// ProcessReader reads all data from src into a buffer, checks the size,
// and returns the content. This pattern is common when you need to
// inspect a stream before committing to expensive processing.
func ProcessReader(src io.Reader) ([]byte, error) {
// Buffer grows as needed. No pre-allocation required.
var buf bytes.Buffer
// io.Copy reads from src and writes to buf in chunks.
// It handles the loop and buffer management for you.
_, err := io.Copy(&buf, src)
if err != nil {
return nil, err
}
// Check the total size after reading.
if buf.Len() > 1024 {
return nil, fmt.Errorf("data too large: %d bytes", buf.Len())
}
// Return a copy of the data.
// The caller owns this slice; it won't be affected by future writes to buf.
return buf.Bytes(), nil
}
The convention here is to return a copy of the bytes. The caller gets ownership. If you return the slice directly, the caller might hold a reference to the buffer's internal memory. If the buffer is reused later, the caller's data could change unexpectedly.
Always check errors from Write and Read. bytes.Buffer rarely returns errors, but the interface contract requires it. If you swap the buffer for a different writer later, errors might occur. Write defensive code.
if err != nil is verbose by design. The community accepts the boilerplate because it makes the unhappy path visible.
Pitfalls and conventions
The slice returned by Bytes is a view, not a copy. It points to the buffer's internal storage. If you write more data, the buffer might reallocate, invalidating the slice. Or it might overwrite the data. Always copy the slice if you need to keep it after mutating the buffer.
// Get a slice view of the buffer content.
data := buf.Bytes()
// Copy the slice if you need to store it or return it.
// make creates a new slice, and copy moves the data.
safeCopy := make([]byte, len(data))
copy(safeCopy, data)
If you try to pass a bytes.Buffer where a string is expected, the compiler rejects it with cannot use buf (variable of type bytes.Buffer) as string value in argument. You must call .String(). The compiler won't convert automatically.
bytes.Buffer is not safe for concurrent use. If multiple goroutines write to the same buffer, you get data races. Use a mutex or separate buffers per goroutine. The worst goroutine bug is the one that never logs.
Don't pass a *bytes.Buffer if you only need to read. Pass a *bytes.Buffer when you need to write. The pointer allows the function to modify the buffer. If you pass by value, the function gets a copy, and changes are lost.
The receiver name is usually one or two letters matching the type: (b *Buffer) Write(...), NOT (this *Buffer) or (self *Buffer). This is a Go convention. Keep it consistent.
Decision matrix
Use bytes.Buffer when you need to read and write bytes, or when you're working with io.Reader and io.Writer interfaces. Use strings.Builder when you are only building strings and never need byte-level access. It avoids the byte-to-string conversion overhead. Use a pre-allocated []byte slice when you know the exact size upfront and want maximum control over memory layout. Use io.Discard when you need to consume a stream but don't care about the data. It implements io.Writer and throws everything away. Use a bytes.Buffer with Reset when you want to reuse memory across multiple operations without reallocating.
Buffers are cheap. Concurrency is not.