Running external commands in Go
You are building a CLI tool that needs to invoke git status to check the repository state. Or perhaps a backend service that triggers a data processing script written in Python. Go does not have a system() call that just runs a string and hopes for the best. You have to be explicit about what you are running, where it runs, and how you handle the output. This explicitness prevents security holes and gives you full control over the subprocess lifecycle.
The Cmd struct is a blueprint
The os/exec package provides the Command function, which returns a *exec.Cmd struct. This struct represents the command before it starts. It holds the binary path, arguments, environment variables, working directory, and stream redirections. Nothing happens when you create the struct. You are building a configuration object. This separation of configuration and execution is a core Go pattern. It allows you to inspect, modify, and validate the command before committing resources.
Think of the Cmd struct like a flight manifest. You list the destination, the passengers, and the cargo. The plane does not take off until the captain gives the order. In Go, the "take off" is calling Run, Start, or one of the output methods. This design makes it easy to unit test command logic by mocking the Cmd struct or by checking its fields before execution.
The Cmd struct is a blueprint. Nothing happens until you ask it to run.
Minimal example
Here is the simplest way to run a command and grab everything it prints. The CombinedOutput method runs the command, waits for it to finish, and captures both standard output and standard error in a single byte slice.
package main
import (
"fmt"
"os/exec"
)
func main() {
// Create a command struct for 'ls -l'.
// The first argument is the binary name, subsequent arguments are flags.
cmd := exec.Command("ls", "-l")
// CombinedOutput runs the command and captures both stdout and stderr.
// It blocks until the command finishes.
output, err := cmd.CombinedOutput()
// Check for execution errors immediately.
if err != nil {
fmt.Printf("failed to run command: %v\n", err)
return
}
// Print the captured output as a string.
fmt.Print(string(output))
}
When you call exec.Command, Go does not start the process yet. It builds the configuration. When you call CombinedOutput, Go forks a new process, sets up pipes for standard output and standard error, waits for the process to exit, and collects all the bytes. If the process exits with a non-zero status, CombinedOutput returns an error. The error type is *exec.ExitError, which contains the exit code and any output that was captured. This design ensures you always have access to the output even when the command fails, which is crucial for debugging.
How execution works under the hood
Go abstracts the operating system details for you. On Unix-like systems, os/exec uses the fork and exec system calls. fork creates a copy of the current process, and exec replaces that copy with the new program. On Windows, it uses CreateProcess. The Cmd struct normalizes these differences so your code works everywhere.
One surprising detail is that exec.Command does not invoke a shell by default. It calls the binary directly. This is a security feature. If you pass a string containing user input to a shell, a malicious user could inject commands like ; rm -rf /. By bypassing the shell, Go eliminates this class of vulnerability. You must split arguments manually. If you write exec.Command("ls -l"), Go looks for a binary literally named ls -l, which does not exist. The runtime will fail with an error like exec: "ls -l": executable file not found in $PATH. Always pass the binary and arguments as separate strings.
Realistic example with context
Real applications rarely run commands in a vacuum. You need timeouts, cancellation, and environment control. The context package is the standard way to manage lifecycles in Go. You should always pass a context to long-running operations, including external commands. The CommandContext function creates a command bound to a context. If the context expires or is cancelled, Go kills the process.
Here is how you run a command with a timeout using context, which prevents your Go program from hanging forever.
package main
import (
"context"
"fmt"
"os/exec"
"time"
)
func main() {
// Create a context with a 2-second timeout.
// This prevents the command from running indefinitely.
ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second)
defer cancel()
// CommandContext binds the command to the context.
// If the context expires, the process is killed.
cmd := exec.CommandContext(ctx, "sleep", "10")
// CombinedOutput runs the command and captures output.
// It returns an error if the context is cancelled.
output, err := cmd.CombinedOutput()
if err != nil {
fmt.Printf("command failed: %v\n", err)
return
}
fmt.Printf("output: %s\n", output)
}
The context.Context parameter always goes first in functions that wrap commands. This is a universal Go convention. It signals to callers that the function respects cancellation and deadlines. When you use CommandContext, the error returned on timeout is an *exec.ExitError with a specific exit code indicating the process was killed. You can inspect err.ExitCode() to distinguish between a timeout and a command failure.
Never trust a subprocess to finish on time. Always set a deadline.
Piping input and output
Sometimes you need to feed data into a command or read its output as it produces it. The convenience methods like CombinedOutput are great for simple cases, but they buffer everything in memory. For large outputs or interactive commands, you need streams. The Cmd struct has Stdin, Stdout, and Stderr fields that you can set to io.Reader or io.Writer values. You can also use StdinPipe, StdoutPipe, and StderrPipe to get connected pipes.
Here is how you pipe data into a command using StdinPipe, which is necessary when the input is dynamic or large.
package main
import (
"fmt"
"os/exec"
)
func main() {
// Create a command that reads from stdin.
cmd := exec.Command("cat")
// StdinPipe creates a writer connected to the command's stdin.
stdin, err := cmd.StdinPipe()
if err != nil {
fmt.Printf("pipe error: %v\n", err)
return
}
// Start runs the command asynchronously.
if err := cmd.Start(); err != nil {
fmt.Printf("start error: %v\n", err)
return
}
// Write data to the pipe.
stdin.Write([]byte("data\n"))
// Close signals end of input.
stdin.Close()
// Wait blocks until the command exits.
cmd.Wait()
}
The Start method launches the process without waiting. This allows you to write to the stdin pipe while the process is running. You must call Wait eventually to reap the process and check the exit status. If you forget to call Wait, the process becomes a zombie on Unix systems. The Run method is a convenience wrapper that calls Start and then Wait. Use Start and Wait only when you need asynchronous interaction. For most cases, Run or CombinedOutput is simpler and safer.
Streams are the lifeblood of Unix tools. Go gives you the pipes; you provide the logic.
Pitfalls and errors
Several things can go wrong when executing external commands. Understanding these pitfalls saves hours of debugging.
If you pass a command string with spaces as a single argument, the shell won't parse it. exec.Command does not invoke a shell by default. You must split arguments manually. The runtime will fail with exec: "ls -l": executable file not found in $PATH if you make this mistake. Always pass the binary and arguments as separate strings.
The Output method captures standard output and returns standard error to the process's stderr. If the command prints to stderr, you won't see it in the byte slice. Use CombinedOutput when you need to capture everything, or set cmd.Stderr explicitly if you want to separate streams. The CombinedOutput method is a wrapper that sets cmd.Stdout and cmd.Stderr to the same pipe internally.
The error returned by Run or Output is often an *exec.ExitError. You can check the exit code using err.ExitCode(). This is useful when a non-zero exit code is expected behavior, like grep returning 1 when no match is found. You can type assert the error to *exec.ExitError to access the exit code and output.
Environment variables are inherited by default. The child process gets a copy of the parent's environment. You can override this by setting cmd.Env. If you set cmd.Env, you must provide the complete environment, including PATH. If you omit PATH, the command won't find binaries. The convention is to append or modify os.Environ() rather than replacing it entirely.
The working directory defaults to the parent process's directory. Set cmd.Dir to change it. This is safer than changing the directory in the parent process, which affects all goroutines.
Shell injection is a silent killer. Avoid the shell unless you have no choice. If you need shell features like globbing or pipes, use exec.Command("sh", "-c", script). Never interpolate user input into the script string. Validate and sanitize input rigorously.
Decision matrix
Use exec.Command with Run when you only care about success or failure and don't need the output.
Use exec.Command with Output when you need standard output and can ignore standard error.
Use exec.Command with CombinedOutput when you need to capture both standard output and standard error in a single call.
Use exec.CommandContext when the command might hang and you need a timeout or cancellation mechanism.
Use exec.Command with Start and Wait when you need to interact with the process streams asynchronously or pipe data dynamically.
Use a shell wrapper like sh -c only when you absolutely need shell features like globbing or pipes, and never with user input.
Pick the method that matches your data flow. Don't capture output you don't need.