The assembly line of concurrency
You have a batch of work to do. Maybe you need to resize a thousand images, parse a massive log file, or fetch data from a dozen APIs. Doing the work one item at a time in a single goroutine is safe, but it leaves your CPU cores sitting idle. You want to split the work across multiple goroutines.
The challenge is coordination. If you launch ten goroutines, how do they share the work? How do they report results back? How do you know when everything is finished without guessing or sleeping for an arbitrary amount of time?
Go solves this with channels. Channels are the pipes that carry data between goroutines. By combining channels with goroutines, you can build structured concurrency patterns that handle distribution, aggregation, and flow control automatically. These patterns have names: Generator, Fan-Out, Fan-In, and Pipeline. They are not special keywords in the language. They are idioms built from the basic primitives you already know.
Think of a factory assembly line. Raw materials arrive at one end. Workers grab materials, process them, and pass them down the line. Finished products come out the other end. The conveyor belt is the channel. The workers are the goroutines. The pattern of how materials move determines the structure of your program.
Generator: feeding the line
A generator is a goroutine that produces data and sends it into a channel. It runs independently and pushes values as they become available. The generator is responsible for closing the channel when it is done. Closing the channel is the signal that no more data is coming.
Without the close signal, receivers will block forever, waiting for a value that never arrives. This causes goroutine leaks. The receiver goroutine stays alive, consuming memory, even though the program is effectively stuck.
package main
import (
"fmt"
)
// Generate produces a stream of integers from the input slice.
// It sends each value into the returned channel and closes the channel when done.
func Generate(nums ...int) <-chan int {
out := make(chan int)
go func() {
// Iterate over the input and send each value to the channel.
for _, n := range nums {
out <- n
}
// Close the channel to signal that no more values will be sent.
// This allows receivers using range to exit their loops.
close(out)
}()
return out
}
func main() {
// Create a generator channel.
c := Generate(1, 2, 3)
// Range over the channel.
// Range blocks until the channel is closed.
for n := range c {
fmt.Println(n)
}
}
The range loop on a channel is the standard way to consume a generator. It pulls values one by one. When the channel is closed and empty, range breaks out of the loop. This is clean and explicit. You do not need to check a boolean flag or pass a done channel. The channel state itself carries the information.
Close the channel when the producer is done. Never close a channel you are sending to.
Fan-Out: splitting the work
Fan-out is the pattern where multiple goroutines read from a single channel. One source feeds many workers. This is useful when you have a backlog of tasks and want to process them in parallel.
The Go runtime distributes values from the channel to the waiting goroutines. If three goroutines are blocked on range c, and a value arrives, the scheduler picks one of them to receive it. The choice is not guaranteed to be round-robin, but it is fair over time. This distribution happens automatically. You do not need to write locking logic to decide which goroutine gets the next item.
package main
import (
"fmt"
"sync"
)
// Worker reads from the input channel and processes values.
// It prints the ID and the value it receives.
func Worker(id int, in <-chan int, wg *sync.WaitGroup) {
// Defer Done to ensure the WaitGroup is decremented when the worker exits.
defer wg.Done()
// Range over the input channel.
// This blocks until a value is available or the channel is closed.
for n := range in {
fmt.Printf("Worker %d processed %d\n", id, n)
}
}
func main() {
// Create a generator channel.
c := Generate(1, 2, 3, 4, 5, 6)
// Create a WaitGroup to track worker completion.
var wg sync.WaitGroup
// Launch three workers.
// Each worker reads from the same channel c.
for i := 1; i <= 3; i++ {
wg.Add(1)
go Worker(i, c, &wg)
}
// Wait for all workers to finish.
wg.Wait()
}
The sync.WaitGroup here tracks the workers, not the data. It ensures main waits for the goroutines to exit. The workers exit when the channel closes. The generator closes the channel after sending all values. The range loops in the workers detect the closure and stop. The WaitGroup counts down to zero, and main proceeds.
Fan-out scales with the number of workers. If you have more work than workers, the workers stay busy. If you have more workers than work, the idle workers block on the channel until data arrives. The channel balances the load.
WaitGroups track completion. Channels move data.
Fan-In: merging results
Fan-in is the reverse of fan-out. Multiple goroutines send data into a single channel. This is useful when you have multiple producers generating results and you want to aggregate them into one stream.
Fan-in is harder to implement correctly than fan-out. The challenge is closing the output channel. You cannot close the output channel until all producers are done. If you close it early, a producer might try to send a value and trigger a panic. If you never close it, the consumer blocks forever.
The standard solution uses a sync.WaitGroup to track the producers. A separate goroutine waits for the WaitGroup to reach zero, then closes the output channel.
package main
import (
"fmt"
"sync"
)
// Merge combines multiple input channels into a single output channel.
// It launches a goroutine for each input channel to copy values to the output.
// It closes the output channel when all input channels are exhausted.
func Merge(cs ...<-chan int) <-chan int {
var wg sync.WaitGroup
out := make(chan int)
// Start a goroutine for each input channel.
for _, c := range cs {
wg.Add(1)
go func(in <-chan int) {
// Defer Done to decrement the WaitGroup when this goroutine finishes.
defer wg.Done()
// Copy all values from the input channel to the output channel.
for n := range in {
out <- n
}
}(c)
}
// Launch a goroutine to close the output channel when all producers are done.
go func() {
wg.Wait()
close(out)
}()
return out
}
func main() {
// Create two generator channels.
c1 := Generate(1, 2, 3)
c2 := Generate(4, 5, 6)
// Merge them into a single channel.
merged := Merge(c1, c2)
// Consume the merged channel.
for n := range merged {
fmt.Println(n)
}
}
Notice the closure pattern in the loop. The goroutine captures the loop variable c by passing it as an argument to the anonymous function. This is a safe practice. In older versions of Go, loop variables were reused across iterations, causing bugs where all goroutines read from the last channel. Go 1.22 fixed this by creating a new variable for each iteration, but passing the variable as an argument remains the clearest way to document intent. It shows exactly which value the goroutine uses.
The Merge function returns a channel that behaves like a generator. The consumer does not know how many producers are behind it. It just ranges over the output. The complexity of tracking producers is hidden inside Merge.
Merge the streams when you need a single view of multiple sources.
Pipeline: chaining stages
A pipeline chains generators, fan-outs, and fan-ins together. The output of one stage becomes the input of the next. This allows you to build complex data processing flows where each stage has a single responsibility.
Pipelines naturally support backpressure. If a downstream stage is slow, it consumes from the channel slowly. The upstream stage blocks on sends until the downstream stage is ready. This prevents the upstream stage from overwhelming the downstream stage with data. The channel size controls the buffer. An unbuffered channel creates tight coupling. A buffered channel allows some decoupling.
package main
import (
"fmt"
)
// Squarer takes a channel of integers and returns a channel of squared integers.
// It reads from the input, squares each value, and sends it to the output.
func Squarer(in <-chan int) <-chan int {
out := make(chan int)
go func() {
for n := range in {
out <- n * n
}
close(out)
}()
return out
}
// Printer consumes a channel of integers and prints them.
func Printer(in <-chan int) {
for n := range in {
fmt.Println(n)
}
}
func main() {
// Stage 1: Generate numbers.
nums := Generate(1, 2, 3, 4, 5)
// Stage 2: Square the numbers.
squared := Squarer(nums)
// Stage 3: Print the results.
Printer(squared)
}
The pipeline runs concurrently. Generate sends a value. Squarer receives it, squares it, and sends it. Printer receives it and prints it. If Printer is slow, Squarer blocks on out <- n * n. If Squarer blocks, Generate blocks on out <- n. The entire pipeline slows down to match the slowest stage. This is efficient. You do not need to manage buffers or thread pools manually. The channels coordinate the flow.
Backpressure is a feature, not a bug. Let the slow consumer throttle the fast producer.
Pitfalls and errors
Channel patterns are powerful, but they have sharp edges. The most common errors come from misunderstanding when to close channels and how range behaves.
If you try to send a value on a closed channel, the program panics with panic: send on closed channel. This happens if you close a channel and a goroutine is still trying to send to it. Always ensure no goroutine is sending before you close. The sync.WaitGroup pattern in Fan-In prevents this by waiting for all senders to finish.
If you try to close a channel twice, the program panics with panic: close of closed channel. This happens if you have multiple goroutines trying to close the same channel. Only the owner of the channel should close it. In Fan-In, the separate goroutine that calls wg.Wait() and then close(out) ensures only one goroutine closes the output.
If you forget to close a channel, the program may deadlock. The receiver blocks on range, waiting for a value. The sender has finished but did not close the channel. No more values will come. The receiver waits forever. The runtime detects this and reports fatal error: all goroutines are asleep - deadlock!. Always close channels when the producer is done.
If you use a buffered channel and fill it, sends block until a receiver reads. If you use an unbuffered channel, sends block until a receiver is ready. This is the same blocking behavior, just with different timing. Choose buffered channels when you want to allow bursts of data. Choose unbuffered channels when you want strict synchronization.
The worst goroutine bug is the one that never logs. If a goroutine leaks, it stays in memory. Use context.Context to cancel long-running goroutines. Pass the context as the first argument to your generator functions. Check for cancellation before sending. This gives you a way to shut down the pipeline if an error occurs or the user cancels the request.
Context is plumbing. Run it through every long-lived call site.
Decision matrix
Use a generator when you need to produce a stream of data for other goroutines to consume.
Use fan-out when you have a single source of work and multiple independent workers that can process items in parallel.
Use fan-in when multiple producers are generating results and you need to aggregate them into a single stream for the next stage.
Use a pipeline when you have a sequence of transformations where the output of one stage becomes the input of the next.
Use a buffered channel when you want to decouple the producer and consumer to allow bursts of data without immediate blocking.
Use an unbuffered channel when you need strict synchronization between the sender and receiver.
Use sync.WaitGroup when you need to wait for multiple goroutines to finish before proceeding.
Use context.Context when you need to cancel or timeout long-running pipeline stages.
Goroutines are cheap. Channels are the glue.