How to Read an Entire File into Memory in Go
You have a configuration file sitting on disk. Your program needs to parse it, but first it needs the bytes. You could open the file, create a buffer, loop until EOF, handle errors on every read, and close the handle. Or you could ask the standard library to do the heavy lifting in one line. Go gives you os.ReadFile. It reads the whole file, returns a byte slice, and handles the file descriptor lifecycle for you.
What os.ReadFile actually does
os.ReadFile is a convenience wrapper around lower-level file operations. It opens the file, reads every byte until the end, closes the file, and hands you a []byte. The function name is slightly misleading if you are used to other languages. It does not read a "file" in the sense of a stream. It reads the contents of a file into memory. The result is a byte slice, which you can convert to a string or process as binary data.
This function is designed for files that fit comfortably in memory. If the file is larger than your available RAM, this function will panic or cause the system to thrash. It is not a streaming tool. It is a bulk loader.
os.ReadFile is a convenience, not a streaming tool.
Minimal example
Here is the simplest way to load a file. One call gets the data, error handling is explicit, and the file closes even if something goes wrong.
package main
import (
"fmt"
"os"
)
func main() {
// ReadFile loads the entire file into a byte slice.
// It handles opening, reading, and closing the file automatically.
data, err := os.ReadFile("config.json")
if err != nil {
// Handle the error immediately. Go makes error paths visible.
fmt.Fprintf(os.Stderr, "failed to read config: %v\n", err)
os.Exit(1)
}
// Convert bytes to string for display.
// The byte slice contains raw data; string conversion is cheap.
fmt.Println(string(data))
}
Walk through the execution
When you call os.ReadFile, the runtime performs several steps behind the scenes. First, it opens the file using the operating system's file API. If the path does not exist or permissions are denied, the function returns an error immediately. The error message looks like open config.json: no such file or directory.
Next, the function allocates a byte slice large enough to hold the file contents. The allocation size depends on the file size reported by the OS. If the OS cannot determine the size, os.ReadFile grows the slice dynamically as it reads. The function reads data in chunks, copying each chunk into the slice until the end of the file is reached.
Finally, it closes the file descriptor. This close happens even if a read error occurs partway through, so you never leak file handles. The function returns the filled byte slice and any error that occurred. If everything succeeded, the error is nil.
The function handles the lifecycle. You get the data.
Realistic example: loading a config
Here is a pattern you will see in real applications. You read the file, then parse the bytes into a struct. This example shows how to wrap errors to provide context.
Start with the data structure you want to populate.
// Config holds the application settings.
type Config struct {
Port int `json:"port"`
Debug bool `json:"debug"`
}
Now load the file and parse it. The function reads the bytes, then unmarshals the JSON.
import (
"encoding/json"
"fmt"
"os"
)
// loadConfig reads the file and parses JSON.
// It wraps errors to indicate whether the failure was I/O or parsing.
func loadConfig(path string) (Config, error) {
var cfg Config
// ReadFile loads the entire file into memory.
// This is appropriate for small configuration files.
data, err := os.ReadFile(path)
if err != nil {
return cfg, fmt.Errorf("read: %w", err)
}
// Unmarshal converts JSON bytes to the struct.
// The json tags map JSON keys to struct fields.
if err := json.Unmarshal(data, &cfg); err != nil {
return cfg, fmt.Errorf("parse: %w", err)
}
return cfg, nil
}
Go requires you to handle errors explicitly. The compiler rejects the program with declared and not used if you assign the error to a variable and never check it. You can discard the error with an underscore if you truly do not care, but for file I/O, ignoring errors is a recipe for silent failures. Always check the error. If the read fails, the byte slice might be empty or partial. Do not assume success.
Pitfalls and edge cases
The biggest pitfall is file size. os.ReadFile allocates a slice for the entire file. If you pass a 10GB log file, the program tries to allocate 10GB of RAM. The allocation fails, or the system starts swapping, and performance collapses. Use os.ReadFile only for files that are small relative to available memory. Configuration files, templates, and small data dumps are fine. Video files, databases, and large logs are not.
Some developers feel compelled to check the file size before reading. They call os.Stat to get the size, compare it to a limit, and then call os.ReadFile. This pattern adds an extra system call and introduces a race condition. The file size can change between the stat and the read. If the file grows, os.ReadFile might allocate more than you expected. If the file shrinks, you might have allocated too much. In most cases, the extra check is unnecessary overhead. os.ReadFile handles size detection efficiently. If you have a strict memory budget, use io.LimitReader instead of pre-checking the size. io.LimitReader enforces the limit during the read, so no race condition exists.
Converting the result to a string creates a copy. The expression string(data) allocates a new string and copies the bytes from the slice. This copy ensures the string is immutable and safe to use after the slice goes out of scope. In Go, strings are immutable. If the conversion did not copy, modifying the underlying byte slice would change the string, which breaks the language guarantees. The copy is fast for small files. For large files, the double allocation (slice plus string) doubles the memory pressure. If you only need the string, you cannot avoid the copy. If you need both, keep the slice and convert only when necessary.
os.ReadFile follows symbolic links. If you pass a path that points to a symlink, the function reads the target file. If the symlink is broken or points to a directory, the function returns an error. This behavior matches the standard Unix semantics for reading files. If you need to read the symlink itself (the path it points to), use os.Readlink instead. ReadFile is for content, not metadata.
You can call os.ReadFile from multiple goroutines safely. The function does not use global state. Each call opens its own file descriptor and manages its own memory. If you need to read many files in parallel, spawn a goroutine for each file. Just be careful about the total memory usage. Reading ten 100MB files in parallel requires 1GB of RAM. Use a worker pool or semaphore if you have many files and limited memory.
Older Go code often uses ioutil.ReadFile. The ioutil package was deprecated in Go 1.16. All functions moved to io or os. ReadFile moved to os. If you see ioutil.ReadFile in a legacy codebase, replace it with os.ReadFile. The behavior is identical. The move reflects Go's philosophy of keeping packages focused. os handles file system operations; io handles streams.
os.ReadFile does not accept a context.Context. You cannot cancel the read operation. If the read is slow, the call blocks until completion. If you need cancellation, you must use os.Open and read with a context-aware reader, or run the read in a goroutine and select on a done channel. For most file reads, cancellation is not needed. Disk I/O is usually fast enough. If you are reading from a network file system that might hang, consider the timeout implications.
Why not just use os.Open and io.ReadAll? You can. The result is the same. os.ReadFile saves you from writing the boilerplate. It also guarantees the file closes. If you write f, err := os.Open(path); data, err := io.ReadAll(f), you must remember to call f.Close(). If you forget, you leak file descriptors. On Linux, the limit is often 1024 or 4096 open files per process. Leaking descriptors causes too many open files errors. os.ReadFile eliminates this class of bug.
Memory is cheap, but not infinite. Check the file size before you read.
When to use os.ReadFile
Use os.ReadFile when the file fits in memory and you want the simplest code to get all bytes. Use os.Open plus a reader when the file is large or you need to process data in chunks to keep memory usage low. Use io.LimitReader when reading from an untrusted source and you need to prevent a malicious file from exhausting memory. Use //go:embed when the file should be compiled into the binary so the program runs without external assets.
Pick the tool that matches the data size. Small file, one call. Large file, stream it.