The plumbing of every CLI tool
You build a command-line tool that processes text. It works perfectly when you run it interactively. Then you pipe the output into another program, and the whole pipeline collapses. The downstream tool chokes because your error messages got mixed into the data stream. Or you try to read input, but the program hangs forever waiting for a newline that never arrives.
Standard streams are the foundation of Unix-style tools. Go exposes them as os.Stdin, os.Stdout, and os.Stderr. These aren't magic functions. They are ordinary file handles that let you read from the input pipe, write results to the output pipe, and send diagnostics to a separate error channel. Understanding how they work prevents pipeline breakage and makes your tools composable.
Streams are just files
In Go, os.Stdin, os.Stdout, and os.Stderr are package-level variables of type *os.File. They represent file descriptors 0, 1, and 2 respectively. This means you can read from stdin, write to stdout, and write to stderr using the exact same methods you use for regular files.
The type is *os.File, which implements io.Reader and io.Writer. This is the key insight: streams behave like files. You can pass them to functions that expect an io.Reader, wrap them in buffers, and chain them with other I/O operations. The distinction between a file on disk and a standard stream is only the underlying file descriptor. The Go API treats them identically.
One surprising detail: fmt.Println is just a wrapper around os.Stdout. When you call fmt.Println("hello"), the runtime writes to os.Stdout behind the scenes. If you want to write to a different destination, you use fmt.Fprintln and pass the target explicitly. This design keeps the API simple while giving you full control when you need it.
Minimal example: read and echo
Here's the simplest interaction: prompt the user, read a line from stdin, and echo it back to stdout.
package main
import (
"bufio"
"fmt"
"os"
)
func main() {
// Scanner buffers reads from stdin for efficiency
scanner := bufio.NewScanner(os.Stdin)
// Prompt the user; fmt.Print writes to os.Stdout by default
fmt.Print("Enter a name: ")
if scanner.Scan() {
// Fprintln writes explicitly to stdout, adding a newline
fmt.Fprintln(os.Stdout, "Hello,", scanner.Text())
} else if err := scanner.Err(); err != nil {
// Report read errors to stderr so they don't pollute output
fmt.Fprintln(os.Stderr, "read error:", err)
}
}
The bufio.Scanner wraps os.Stdin and reads data in chunks. Scan advances to the next token and returns true until it hits EOF or an error. Text returns the current token as a string. The default split function breaks on newlines, so Scan reads one line at a time.
When you run this program interactively, os.Stdin blocks until you type something and press Enter. The scanner buffers the input, Scan returns true, and Text gives you the line. If you pipe input like echo "Alice" | go run main.go, the scanner reads the piped data, hits EOF immediately after the line, and Scan returns false on the next call. The program exits cleanly.
Realistic example: a filter with separation
A production tool needs to separate data from diagnostics. Data goes to stdout so it can be piped. Errors and warnings go to stderr so they bypass the pipe and still reach the user. Here's a filter that reads lines, writes matches to stdout, and sends skipped lines to stderr.
package main
import (
"bufio"
"fmt"
"os"
"strings"
)
// Filter writes lines containing pattern to stdout
func Filter(pattern string) {
scanner := bufio.NewScanner(os.Stdin)
// Loop until EOF; Scan returns false when input ends
for scanner.Scan() {
line := scanner.Text()
if strings.Contains(line, pattern) {
// Data output goes to stdout for piping
fmt.Fprintln(os.Stdout, line)
} else {
// Diagnostics go to stderr to avoid breaking downstream parsers
fmt.Fprintln(os.Stderr, "skipped:", line)
}
}
// Always check scanner errors after the loop
if err := scanner.Err(); err != nil {
fmt.Fprintf(os.Stderr, "read error: %v\n", err)
}
}
func main() {
if len(os.Args) < 2 {
// Usage errors go to stderr with a non-zero exit code
fmt.Fprintln(os.Stderr, "usage: filter <pattern>")
os.Exit(1)
}
Filter(os.Args[1])
}
The loop processes each line. Matching lines go to os.Stdout. Non-matching lines go to os.Stderr. If you run go run main.go foo < input.txt, the matches appear in the terminal while skipped lines also appear but are tagged. If you pipe the output, only matches pass through. The stderr stream remains visible to the user but doesn't interfere with the data pipeline.
After the loop, scanner.Err() checks for read errors. The scanner might fail if the input is too large for the buffer or if the underlying file descriptor has an issue. Checking the error after the loop is standard Go practice. The compiler won't force you to check it, but skipping the check hides real failures.
Pitfalls: blocking, buffering, and globals
Standard streams have quirks that trip up new Go developers. Knowing these patterns saves debugging time.
Blocking on stdin. If your program reads from os.Stdin but no input is available, it blocks. In a terminal, this waits for the user. In a pipeline, it waits for the upstream process to write. If the upstream process crashes without closing its output, your program hangs forever. Always design for EOF. If you need a timeout, wrap the read in a goroutine with a context.Context or use os.Stdin.SetReadDeadline.
Scanner token size. bufio.Scanner has a default maximum token size of 64KB. If a single line exceeds this limit, Scan returns false and Err returns bufio.Scanner token too long. The program stops reading. If you expect large lines, increase the buffer with scanner.Buffer(buf, maxSize) or switch to bufio.Reader and use ReadLine.
Never close standard streams. os.Stdin, os.Stdout, and os.Stderr are global file handles. Closing them breaks other parts of your program and can confuse the runtime. If you wrap them in a bufio.Scanner or bufio.Writer, the wrapper doesn't close the underlying file. You can safely let the program exit without closing streams. The OS cleans up file descriptors when the process ends.
Mixing buffered and unbuffered writes. fmt.Fprintln writes directly to the file. bufio.Writer buffers writes and flushes when the buffer is full or when you call Flush. If you mix fmt calls with a bufio.Writer on the same stream, output can interleave in unexpected order. Pick one strategy per stream. Use fmt for simple output, or use a bufio.Writer for high-performance writes, but don't mix them.
Compiler errors. If you forget to import os, the compiler rejects the program with undefined: os. If you pass a string where a writer is expected, you get cannot use "text" (untyped string constant) as io.Writer value in argument. These errors are straightforward. The compiler tells you exactly what type it expects.
Stdout is for data. Stderr is for humans. Never mix them.
Design for interfaces, not globals
Hardcoding os.Stdin and os.Stdout in your functions makes them hard to test. You can't easily inject mock input or capture output without rewriting the streams globally, which is fragile and breaks concurrency. The Go solution is to accept io.Reader and io.Writer interfaces instead.
Interfaces describe behavior. io.Reader requires a Read method. io.Writer requires a Write method. *os.File implements both. So does bytes.Buffer, strings.Reader, and os.File. By accepting interfaces, your functions work with files, streams, buffers, and strings interchangeably. This is the "accept interfaces, return structs" mantra in action.
Here's the filter refactored to use interfaces:
package main
import (
"bufio"
"fmt"
"io"
"os"
"strings"
)
// FilterIO reads from r and writes matches to w
func FilterIO(r io.Reader, w io.Writer, pattern string) error {
scanner := bufio.NewScanner(r)
// Loop until EOF; works with any io.Reader
for scanner.Scan() {
if strings.Contains(scanner.Text(), pattern) {
// Write to the provided writer interface
fmt.Fprintln(w, scanner.Text())
}
}
// Return scanner errors to the caller
return scanner.Err()
}
func main() {
if len(os.Args) < 2 {
fmt.Fprintln(os.Stderr, "usage: filter <pattern>")
os.Exit(1)
}
// Pass os.Stdin and os.Stdout to the interface-based function
if err := FilterIO(os.Stdin, os.Stdout, os.Args[1]); err != nil {
fmt.Fprintf(os.Stderr, "error: %v\n", err)
os.Exit(1)
}
}
The FilterIO function takes io.Reader and io.Writer. It doesn't know about os.Stdin or os.Stdout. In main, you pass the standard streams. In tests, you can pass strings.NewReader for input and bytes.Buffer for output. This makes testing trivial. You can verify the output without touching the filesystem or standard streams.
The convention here is clear: functions that do I/O should accept interfaces. The main function is the only place that should reference os.Stdin and os.Stdout directly. This keeps your code modular and testable.
Functions that take a context should respect cancellation and deadlines. If your I/O function is long-running, add context.Context as the first parameter. Check ctx.Done() periodically or use ctx.Deadline() to set timeouts on reads. Context is plumbing. Run it through every long-lived call site.
Decision: when to use what
Use os.Stdin when you need to read from the terminal or a pipe in a simple script. Use os.Stdout when writing data that might be piped to another tool. Use os.Stderr when writing error messages or logs that should bypass pipes and reach the user. Use io.Reader and io.Writer interfaces when writing functions that should work with files, streams, or buffers interchangeably. Use bufio.Scanner when reading line-oriented text with reasonable line lengths. Use bufio.Reader when you need to handle very large lines or custom parsing. Use the log package when you need timestamps and automatic stderr routing. Use fmt.Fprintln when you want explicit control over the output destination. Use os.Args when you need command-line arguments, not stream data.
Treat streams like files. They are files.