When Go needs to call the outside world
You are building a CLI tool that audits system configuration. The tool needs to read the current user's groups, check the disk usage of a specific mount point, and verify that a background service is running. Go has excellent libraries for file I/O and networking, but these tasks rely on system utilities that are already installed and battle-tested. Writing a Go replacement for df or systemctl adds maintenance burden and risks subtle platform differences. You need your Go program to invoke these external binaries, pass arguments, capture the results, and handle failures gracefully.
The os/exec package bridges the gap between your Go code and the operating system. It lets you spawn external processes, connect their input and output streams, and wait for them to finish. You are not parsing shell commands. You are invoking binaries directly. This approach is safer and more predictable than shelling out to a shell interpreter.
The Cmd struct and execution lifecycle
The package revolves around the Cmd struct. This struct describes an external process before it runs. It holds the program path, arguments, environment variables, working directory, and stream redirections. Nothing happens when you create a Cmd. The struct is just a configuration object. You must call a method like Run, Output, or Start to trigger execution.
This separation of configuration and execution is deliberate. It allows you to build a command, inspect or modify its settings, and then run it. You can reuse the same Cmd logic with different arguments, or check that a binary exists before attempting to run it. The design keeps the API flexible while maintaining clear boundaries between setup and runtime.
The Cmd struct has many fields. You rarely set them all. The community convention is to use the helper functions Command and CommandContext to initialize the struct, then modify only the fields you need. Don't construct a Cmd manually unless you have a specific reason. Also, remember that gofmt handles the formatting. Your code will look consistent with the rest of the ecosystem if you let the tool run.
External processes are black boxes. Treat them as hostile until they prove otherwise.
Minimal example: capture output and check errors
Here is the simplest way to run a command and capture its standard output. The code uses exec.Command to build the command and Output to run it.
package main
import (
"fmt"
"os/exec"
)
func main() {
// Command takes the program name as the first argument,
// followed by separate arguments for each flag or parameter.
cmd := exec.Command("ls", "-l")
// Output runs the command and returns stdout as a byte slice.
// It blocks until the command finishes.
output, err := cmd.Output()
if err != nil {
// The compiler rejects the program if you ignore the error here.
// Always check errors from exec calls.
fmt.Println(err)
return
}
// Convert the byte slice to a string for display.
fmt.Println(string(output))
}
The Output method handles the entire lifecycle. It sets up a pipe to capture standard output, starts the process, waits for it to exit, and returns the captured bytes. If the process exits with a zero status, Output returns the output and a nil error. If the exit status is non-zero, or if the process cannot be started, Output returns an error. The error type is usually *exec.ExitError, which contains the exit code and any stderr output if you configured it.
Capture the output. Check the error. Move on.
What happens under the hood
Calling exec.Command does not run anything yet. The function searches for the executable in the PATH environment variable if you provide a relative name. It resolves the full path and populates the Cmd struct. No process starts yet.
When you call Output, Go creates pipes for stdout and stderr if needed. It calls the OS to start the process. The current goroutine blocks. The OS runs the external program. The program writes to its file descriptors. Go reads from the pipes and buffers the data. When the program exits, the OS signals Go. Go checks the exit code. If the code is zero, Output returns the buffered stdout. If the code is non-zero, Output returns an error.
The error wrapping follows standard Go conventions. The if err != nil check is verbose by design. The community accepts the boilerplate because it makes the unhappy path visible. You should always check the error. If you capture an error and don't use it, the compiler rejects the program with declared and not used. This forces you to handle errors explicitly.
The Cmd struct is a blueprint. Execution happens only when you ask for it.
Realistic example: context, streams, and timeouts
Real code needs context for timeouts, separate error handling, and safe argument passing. External processes can hang, consume resources, or fail silently. You need to control their lifecycle and inspect their output.
Here is how to set up a command with context and stream handling. The code uses CommandContext to tie the process to a context, and StderrPipe to capture error output separately.
package main
import (
"context"
"fmt"
"os/exec"
"time"
)
// RunGzip compresses a file using the system gzip utility.
// It respects context cancellation and captures stderr for logging.
func RunGzip(ctx context.Context, filename string) error {
// CommandContext creates a Cmd that respects the context.
// If the context expires, the process is killed.
cmd := exec.CommandContext(ctx, "gzip", filename)
// StderrPipe creates a pipe for error output.
// This keeps stderr separate from stdout.
stderr, err := cmd.StderrPipe()
if err != nil {
return fmt.Errorf("pipe setup failed: %w", err)
}
// Start launches the process asynchronously.
// The function returns immediately after the process begins.
if err := cmd.Start(); err != nil {
return fmt.Errorf("start failed: %w", err)
}
// Close the pipe when done to release resources.
// This prevents goroutine leaks if the caller returns early.
defer stderr.Close()
The context must be the first parameter by convention. Functions that take a context should respect cancellation and deadlines. CommandContext ensures that if the context is cancelled, the process receives a signal to terminate. This prevents orphaned processes.
Here is how to wait for the process and handle the result. The code reads stderr and waits for the process to exit.
// ... inside RunGzip, after Start ...
// Read stderr to capture error messages from the tool.
// The buffer size is small for demonstration.
buf := make([]byte, 1024)
for {
n, err := stderr.Read(buf)
if n > 0 {
// Log the error output from the external tool.
fmt.Printf("gzip stderr: %s", buf[:n])
}
if err != nil {
break
}
}
// Wait blocks until the process finishes.
// It also cleans up the OS process resources.
if err := cmd.Wait(); err != nil {
// The error contains the exit code and stderr if configured.
return fmt.Errorf("process exited with error: %w", err)
}
return nil
}
The Wait method blocks until the process exits. It also cleans up the OS process resources. If you don't call Wait, the process might become a zombie. The error returned by Wait includes the exit status. You can type-assert the error to *exec.ExitError to inspect the exit code or stderr.
Context is plumbing. Run it through every long-lived call site.
Pitfalls and compiler errors
Arguments must be separate strings. Passing a single string with spaces causes the runtime to look for a binary with spaces in the name. The error message is exec: "ls -l": executable file not found in $PATH. This happens because the system treats the whole string as the program name. Always split arguments.
If you forget to capture the loop variable in a closure, the compiler rejects the program with loop variable i captured by func literal. This became a hard error in Go 1.22. Be careful when spawning goroutines in loops.
Another pitfall is ignoring stderr. Output captures stdout only. If the command fails and writes to stderr, you get an error but no details. Use StderrPipe to capture error output. The compiler complains with imported and not used if you import a package but don't use it. If you import context but don't use it, the build fails.
Context leaks are common. If you use CommandContext, the context must be cancelled or the process might remain running. The context sends a signal to kill the process on cancellation. If you don't use context, a hanging process can block your goroutine forever. The worst goroutine bug is the one that never logs.
Arguments are not shell strings. Split them or break your build.
Decision: when to use exec vs alternatives
Use exec.Command with .Output() when you need to run a quick command and capture its stdout. Use exec.CommandContext when the command might hang or you need a timeout. Use cmd.Start() and cmd.Wait() when you need to stream input or output while the process runs. Use cmd.Run() when you don't need output and just want to check the exit code. Use a pure Go library instead of os/exec when the functionality is available in the standard library or a trusted third-party package. Use os/exec.LookPath when you need to verify a binary exists before attempting to run it.
Prefer Go code over shell scripts. Call out only when you must.