Reading stdin and writing stdout
You are building a command-line tool that transforms text. The user pipes a log file into it, or types commands directly into the terminal. Either way, your program needs to accept data, process it, and send results back out. Go handles this through three standard streams: standard input for incoming data, standard output for results, and standard error for diagnostics. You do not need special libraries to make this work. The standard library gives you everything, and it follows a consistent pattern.
The io.Reader and io.Writer contract
Think of standard streams like universal plumbing adapters. os.Stdin, os.Stdout, and os.Stderr are just file handles that implement two interfaces: io.Reader and io.Writer. Any function that accepts an io.Reader can read from a terminal, a pipe, a file, or a network socket without changing a single line of code. The interface abstracts the source. You write your logic once, and it works everywhere.
The io.Reader contract is deliberately minimal. It asks for a byte slice, fills it with data, and returns how many bytes it wrote plus an error. When the stream ends, it returns io.EOF. You will often read fewer bytes than requested. The caller handles partial reads. This design keeps I/O operations composable and predictable. A function that reads from a reader does not care whether the data comes from a file, a pipe, or memory.
The io.Writer contract mirrors this. It accepts a byte slice, writes as much as it can, and returns the count plus an error. The fmt package uses both interfaces under the hood. When you call fmt.Fprintln(os.Stdout, "hello"), the fmt package formats the string into bytes and hands them to os.Stdout.Write. You rarely call Read or Write directly. You use higher-level wrappers that handle the contract for you.
Interfaces are accepted, structs are returned. This mantra shapes how Go developers design I/O functions. You pass io.Reader into your processing function, but you return concrete types or errors. The abstraction stays at the boundary.
Minimal example: line-by-line scanning
Here is the standard pattern for reading text line by line. Create a scanner, loop until it stops, and check for errors afterward.
package main
import (
"bufio"
"fmt"
"os"
)
func main() {
// Scanner wraps os.Stdin and handles buffering automatically.
// It splits input by newline characters by default.
scanner := bufio.NewScanner(os.Stdin)
for scanner.Scan() {
// Scan returns true when a new line is successfully loaded.
// Text returns the line content without the trailing newline.
fmt.Fprintln(os.Stdout, scanner.Text())
}
// Always check scanner.Err() after the loop exits.
// It distinguishes between a clean end-of-file and a real I/O failure.
if err := scanner.Err(); err != nil {
fmt.Fprintln(os.Stderr, "error:", err)
os.Exit(1)
}
}
The scanner manages a user-space buffer. It pulls a chunk of data from the kernel into memory, then splits that chunk by the delimiter. The default delimiter is a newline. When you call scanner.Scan, the scanner checks its buffer. If the buffer is empty, it makes a system call to read more data from the kernel. This batching reduces system calls dramatically compared to reading byte by byte.
When the input stream closes, Scan returns false. The loop exits. You must call scanner.Err to know why. If the stream ended cleanly, Err returns nil. If a disk error or allocation failure occurred, Err returns that error. Skipping this check is a common mistake. The loop ending does not guarantee success.
Realistic example: processing and error routing
Real tools separate data from diagnostics. This example processes lines, writes results to stdout, and routes warnings to stderr. It also shows how to handle lines that exceed the default buffer size.
package main
import (
"bufio"
"fmt"
"os"
"strings"
)
// ProcessLine transforms a single line of input.
// It returns the transformed string or an error if the line is invalid.
func ProcessLine(line string) (string, error) {
if strings.HasPrefix(line, "SKIP:") {
return "", fmt.Errorf("skipping line: %s", line)
}
return strings.ToUpper(line), nil
}
func main() {
scanner := bufio.NewScanner(os.Stdin)
// Allocate a larger buffer for inputs with very long lines.
// The first argument is the initial buffer, the second is the max capacity.
scanner.Buffer(make([]byte, 1024), 1024*1024)
for scanner.Scan() {
result, err := ProcessLine(scanner.Text())
if err != nil {
// Route warnings to stderr so they never mix with program output.
fmt.Fprintln(os.Stderr, "warning:", err)
continue
}
fmt.Fprintln(os.Stdout, result)
}
if err := scanner.Err(); err != nil {
fmt.Fprintln(os.Stderr, "fatal:", err)
os.Exit(1)
}
}
The scanner buffer defaults to 64 kilobytes. If a single line exceeds that limit, Scan returns false and scanner.Err returns bufio.ErrTooLong. The loop stops immediately. You can resize the buffer with scanner.Buffer if your data contains massive lines. The first argument sets the starting slice, and the second sets the maximum growth limit.
Writing to os.Stderr keeps warnings separate from data. If the user pipes your output to another tool, the warnings will not interfere. cat data.txt | mytool | grep pattern. If mytool prints "Starting..." to stdout, grep sees it. Stderr flows to the terminal by default, even when stdout is piped. This separation is fundamental to Unix philosophy.
Pitfalls and compiler errors
Forgetting to check scanner.Err is a silent failure waiting to happen. The loop might exit because the disk filled up, not because the input ended. Always inspect the error after the loop finishes.
Type mismatches trigger immediate compiler rejections. If you pass os.Stdin to a function expecting a string, the compiler rejects the program with cannot use os.Stdin (variable of type *os.File) as string value in argument. os.Stdin is a reader, not a string. You must read from it first.
Unused imports also fail fast. The compiler complains with imported and not used if you bring in a package but never reference it. Go forces you to keep the dependency graph clean.
Interactive mode introduces blocking behavior. os.Stdin waits indefinitely if no input arrives. The program does not know whether a human is typing or a pipe is feeding data. The community convention for detecting a terminal is to use golang.org/x/term. Calling term.IsTerminal(syscall.Stdin) returns true when a human is typing. This lets you show prompts only in interactive sessions.
Do not pass a *string to functions that need text. Strings are already cheap to pass by value. The compiler will not stop you, but the runtime will allocate unnecessarily. Stick to plain string values for text processing.
Conventions and style
Go follows strict formatting and naming conventions. Trust gofmt. Your code will match the rest of the ecosystem. Most editors run it automatically on save. Do not argue about indentation.
Error handling is verbose by design. if err != nil { return err } makes the failure path impossible to ignore. The community accepts the repetition because it prevents silent crashes. In a CLI, you typically print the error and call os.Exit(1). Exit code zero means success. Any non-zero value signals failure to the shell.
Function names start with a capital letter if they are exported. ProcessLine is exported. main is not exported. It is the entry point. Package names are lowercase. import "bufio" not import "Bufio".
The receiver name in methods is usually one or two letters matching the type. (s *Scanner) Scan() not (this *Scanner) Scan(). This keeps method signatures short and readable.
The underscore discards a value intentionally. result, _ := ... says you considered the second return value and chose to drop it. Use it sparingly with errors. Dropping an error without acknowledging it defeats the purpose of Go's explicit error handling.
Context is plumbing. Run it through every long-lived call site. If your CLI tool spawns background work or talks to external services, pass context.Context as the first parameter. Name it ctx. Respect cancellation and deadlines.
Decision matrix
Use bufio.Scanner when you are processing text line by line and lines fit comfortably in memory.
Use bufio.Reader when you need fine-grained control over byte reading or are parsing a binary format.
Use os.ReadFile when the input comes from a file path argument rather than a stream, and the file size is small enough to load entirely.
Use io.Copy when you just need to pipe data from one stream to another without parsing or transforming it.
Use a custom io.Reader wrapper when you need to transform the stream on the fly, like decompressing gzip data or stripping headers.
Streams are interfaces. Treat them like plumbing. Check scanner.Err. The loop ending is not always success. Pick the reader that matches your data shape.