Fan-out fan-in pattern with channels

The fan-out fan-in pattern distributes work to multiple goroutines and collects results when all are done. In the Go compiler, compileFunctions spawns nBackendWorkers goroutines that pull functions from a shared compilequeue and push closures back into it.

var mu sync.Mutex
var wg sync.WaitGroup
mu.Lock()

for workerId := range base.Flag.LowerC {
	wg.Add(1)
	go func() {
		defer wg.Done()
		var closures []*ir.Func
		for {
			mu.Lock()
			compilequeue = append(compilequeue, closures...)
			if len(compilequeue) == 0 {
				mu.Unlock()
				return
			}
			fn := compilequeue[len(compilequeue)-1]
			compilequeue = compilequeue[:len(compilequeue)-1]
			mu.Unlock()
			ssagen.Compile(fn, workerId, profile)
			closures = fn.Closures
		}
	}()
}

mu.Unlock()
wg.Wait()

The fan-out fan-in pattern with channels splits a big job into smaller pieces and gives them to many workers at once. Each worker grabs a piece, does the work, and if that work creates more pieces, they are added back to the pile for others to grab. The main process waits until the pile is empty and all workers are finished.