---
title: How to Work with Unix Pipes in Go
summary: Create a Unix pipe in Go using os.Pipe() to connect a writer and reader for streaming data between goroutines.
lang: en
slug: how-to-work-with-unix-pipes-in-go
published: 2026-04-17T12:05:42+00:00
updated: 2026-04-25T22:00:13+00:00
canonical: https://www.gofaq.org/en/how-to-work-with-unix-pipes-in-go/
---

# How to Work with Unix Pipes in Go



## The tube between goroutines

You are building a tool that needs to compress a stream of data and send it over the network, or maybe you are launching a subprocess and need to feed it input while reading its output. You do not want to write everything to a temporary file. You want the output of one stage to flow directly into the input of the next, just like `cat file | gzip > out.gz` in the shell. In Go, you get that same streaming behavior with `os.Pipe`.

A pipe is a one-way street in memory. One goroutine writes to the street, another reads from it. The operating system handles the buffering. If the writer is faster, the reader catches up. If the reader is faster, the writer blocks until data arrives. It is a synchronization primitive disguised as I/O.

`os.Pipe` creates a kernel-backed pipe and returns two file descriptors: one for reading, one for writing. The read end implements `io.Reader`. The write end implements `io.Writer`. You can pass these to any function that expects those interfaces, including `io.Copy`, `os/exec` commands, or compression writers.

## Minimal pipe example

Here is the simplest pipe: create it, spawn a writer, read until EOF.

```go
package main

import (
	"fmt"
	"io"
	"os"
)

func main() {
	// os.Pipe creates a connected pipe in the kernel.
	// It returns a reader file and a writer file.
	reader, writer, err := os.Pipe()
	if err != nil {
		panic(err)
	}

	// Run the writer in a goroutine to avoid deadlock.
	// The main goroutine will read, so the writer needs its own thread of execution.
	go func() {
		// Defer close ensures the pipe closes even if the write panics.
		// Closing the writer signals EOF to the reader.
		defer writer.Close()
		writer.Write([]byte("Hello from the pipe"))
	}()

	// ReadAll blocks until the writer closes the pipe or returns an error.
	// It consumes the entire stream into memory.
	data, err := io.ReadAll(reader)
	if err != nil {
		panic(err)
	}
	fmt.Println(string(data))
	// Close the reader to release the file descriptor.
	reader.Close()
}
```

The writer runs in a goroutine because `Write` blocks if the pipe buffer is full, and `ReadAll` blocks if the buffer is empty. If both ran in the same goroutine, the program would deadlock immediately. The `defer writer.Close()` is critical. `io.ReadAll` waits for EOF. EOF happens when the writer closes the pipe. Without that close, the reader hangs forever.

Pipes block until data flows. Always close the writer to unblock the reader.

## How the kernel handles the flow

When you call `os.Pipe`, the OS allocates a circular buffer in kernel space. The size is usually 64KB on Linux, but you should not rely on the exact size. The buffer decouples the writer and reader slightly. The writer can push data into the buffer and return, even if the reader has not started yet. Once the buffer fills, the writer blocks until the reader drains some bytes.

If the reader reads faster than the writer, the reader blocks when the buffer is empty. This blocking is efficient. The OS puts the goroutine to sleep and wakes it when data arrives. No busy-waiting.

Error handling follows the standard Go convention. The community accepts the boilerplate of `if err != nil` because it makes the unhappy path visible. Pipes fail, and you need to know when they do. If you forget to capture the error from `os.Pipe`, the compiler rejects the program with `error returned but not handled`. If you pass the wrong type to a function expecting an `io.Reader`, the compiler complains with `cannot use writer (variable of type *os.File) as io.Reader value in argument`.

Trust the pipe buffer, but respect the close.

## Streaming with io.Copy

Real code rarely reads everything into memory. Use `io.Copy` to stream data from the pipe to a destination without buffering the whole payload. `io.Copy` reads from the source, writes to the destination, and reuses a small internal buffer. It handles chunking automatically.

```go
func streamChunks() {
	// Create a pipe for streaming data between stages.
	pr, pw, err := os.Pipe()
	if err != nil {
		panic(err)
	}

	// Writer goroutine simulates a data source producing chunks.
	go func() {
		defer pw.Close()
		// Write in small chunks to demonstrate streaming behavior.
		// The pipe buffers data in the kernel until the reader consumes it.
		for i := 0; i < 5; i++ {
			pw.Write([]byte(fmt.Sprintf("Chunk %d\n", i)))
		}
	}()

	// io.Copy reads from pr and writes to os.Stdout until EOF.
	// It handles buffering and chunking automatically, making it efficient for large streams.
	if _, err := io.Copy(os.Stdout, pr); err != nil {
		panic(err)
	}
	pr.Close()
}
```

`io.Copy` is the workhorse for moving data between readers and writers. It returns the number of bytes copied and an error. If the writer closes the pipe, `io.Copy` returns `nil` error and the byte count. If the reader closes early, `io.Copy` returns an error. The writer goroutine will see a `write |0: broken pipe` error on the next write attempt. This happens when the consumer stops listening.

The receiver name convention applies here too. If you wrap this logic in a struct method, name the receiver one or two letters matching the type: `(s *Streamer) Stream(...)`, not `(this *Streamer)`. Keep it idiomatic.

## User-space pipes with io.Pipe

Sometimes you do not need the kernel. `io.Pipe` creates a synchronous in-memory pipe that implements `io.Reader` and `io.Writer` interfaces. It runs entirely in user space, managed by goroutines. Use `io.Pipe` when you need to chain pure Go streams or mock I/O for testing.

```go
func main() {
	// io.Pipe creates a synchronous in-memory pipe.
	// It returns io.Reader and io.Writer interfaces, not file descriptors.
	pr, pw := io.Pipe()

	// The writer must run in a goroutine.
	// io.Pipe blocks until both the reader and writer are active.
	go func() {
		defer pw.Close()
		pw.Write([]byte("Data from io.Pipe"))
	}()

	// ReadAll consumes the stream.
	// io.Pipe handles synchronization entirely in user space.
	data, err := io.ReadAll(pr)
	if err != nil {
		panic(err)
	}
	fmt.Println(string(data))
}
```

`io.Pipe` returns interfaces, not `*os.File`. This means you cannot pass the ends to system calls or subprocesses. It is lighter weight for in-process communication. The writer blocks until the reader starts reading, and vice versa. `io.Pipe` is useful when you want to inject a stream into a function that accepts `io.Reader` without creating a kernel object.

`os.Pipe` talks to the kernel. `io.Pipe` stays in Go. Pick the tool that matches your boundary.

## Pitfalls and leaks

Pipes introduce concurrency hazards. The most common bug is a goroutine leak. If you spawn a writer goroutine and the reader returns early without closing the pipe, the writer might block forever on `Write`. The goroutine never exits, and the program holds resources indefinitely. Always ensure the reader closes the pipe or the writer has a cancellation path.

Context is plumbing. If your pipe writer runs for a long time, pass a `context.Context` as the first parameter, conventionally named `ctx`. Functions that take a context should respect cancellation and deadlines. If the context cancels, the writer should stop writing and close the pipe.

Another pitfall is ignoring the error from `Write`. If the reader closes the pipe, `Write` returns an error. If you ignore it, you might continue writing to a dead pipe, wasting CPU cycles. Check the error and break the loop.

The worst goroutine bug is the one that never logs. Add logging to your pipe writer so you can detect when it blocks or fails. If a goroutine hangs, logs help you find the dead end.

## When to use pipes

Use `os.Pipe` when you need a kernel-managed pipe for subprocess I/O or when you require file descriptors that can be passed to system calls.

Use `io.Pipe` when you need a user-space pipe that implements `io.Reader` and `io.Writer` for chaining pure Go streams without kernel overhead.

Use `os/exec` with `StdinPipe` when you are launching an external command and need to write data to its standard input while reading its output.

Use a channel when you are passing typed Go values between goroutines and want the compiler to enforce type safety and synchronization.

Goroutines are cheap. Pipes are not magic.

## Where to go next

- [How to Read and Write ZIP Archives in Go](/en/how-to-read-and-write-zip-archives-in-go/)
- [How to Read and Write Binary Files in Go](/en/how-to-read-and-write-binary-files-in-go/)
- [Limitations of go:embed and How to Work Around Them](/en/limitations-of-goembed-and-how-to-work-around-them/)