How to Write a Custom Iterator in Go

The problem with intermediate slices

You are processing a log file with ten million lines. You need to filter for errors, extract the timestamp, and write the results to a report. In older Go, you often reached for slices. You read lines into a slice, filtered them into a second slice, mapped them into a third, and then iterated over the final slice. Each step allocated memory. Each step copied data. If the file is large, you burn gigabytes of RAM just to hold intermediate lists that exist for a fraction of a second.

You want to process one line at a time. You want to pass the result down a chain of functions without building temporary collections. You want the memory footprint to stay flat regardless of input size. Go 1.23 introduced pull iterators to solve this. The iter.Seq[T] type lets you write lazy, composable pipelines that allocate nothing and run as fast as a hand-written loop.

What iter.Seq actually is

An iterator is a recipe for producing values, not a bag of values. Think of a tap. You don't fill a bucket with all the water in the pipe. You turn the handle, get a cup of water, do something with it, and turn the handle again. The iterator holds the state of where you are in the stream. The iter.Seq[T] type is that tap. It promises to give you values of type T one by one until you ask it to stop or it runs out.

The type signature looks dense at first glance:

type Seq[T any] func(func(T) bool)

This defines Seq[T] as a function type. Your iterator is a function that returns a closure. That closure takes a yield function as an argument. The yield function is the bridge between your iterator and the consumer. When the consumer wants a value, it calls your closure. Your closure calls yield with a value. The consumer receives the value, runs its loop body, and then calls yield again if it wants more. If the consumer breaks, yield returns false, signaling your iterator to stop.

Iterators are pull-based. The consumer drives the flow. The iterator only does work when the consumer asks for the next value. This keeps memory usage constant and allows early termination.

Minimal example

Here's the simplest custom iterator. It yields integers from a start value up to a limit.

package main

import (
	"fmt"
	"iter"
)

// CountUp returns an iterator that yields integers from start up to limit.
func CountUp(start, limit int) iter.Seq[int] {
	return func(yield func(int) bool) {
		// Loop through the range and yield each value.
		for i := start; i <= limit; i++ {
			// yield sends the value to the consumer.
			// It returns false if the consumer stops early.
			if !yield(i) {
				return
			}
		}
	}
}

func main() {
	// The for loop drives the iterator.
	for n := range CountUp(1, 5) {
		fmt.Println(n)
	}
}

The for n := range CountUp(1, 5) syntax works because iter.Seq is a function type. The range clause calls CountUp, gets the closure, creates a yield function, and passes it to the closure. The closure runs, calling yield repeatedly. The loop body executes between each yield call.

Check the yield return value. If you ignore it, your iterator keeps running even after the consumer breaks. That wastes CPU and can leak resources. Always check if !yield(v) { return }.

Iterators are lazy. Check the yield return.

How the control flow works

Understanding who calls whom prevents subtle bugs. The for loop is the driver. The iterator is the supplier.

The for loop calls your iterator function.
Your function returns a closure.
The for loop creates a yield function and calls your closure with it.
Your closure runs. It calls yield(value).
The for loop receives the value, assigns it to the loop variable, and runs the loop body.
When the loop body finishes, control returns to yield, which returns true.
Your closure continues. It calls yield again with the next value.
This repeats until your closure returns or yield returns false.

If the loop body contains a break, the for loop makes the next yield call return false. Your closure sees false and should return immediately. If you don't check the return value, your closure calls yield again. The for loop sees false, stops, but your closure is still running. You've leaked work.

This flow also means you can compose iterators freely. You can pass an iterator into a function that returns another iterator. The composition happens at the function level. No interfaces, no allocations. Just functions calling functions.

Realistic example with state and cleanup

Real iterators often hold state. They might read from a file, query a database, or maintain a cursor. Structs are the natural home for this state. The struct holds the fields, and a method returns the iter.Seq.

Here's an iterator that reads lines from a file. It opens the file, yields lines, and closes the file when done.

package main

import (
	"bufio"
	"fmt"
	"iter"
	"os"
)

// FileLines holds the path to a file for iteration.
type FileLines struct {
	Path string
}

// Seq returns an iterator that yields lines from the file.
// The receiver name r matches the type, following Go convention.
func (r FileLines) Seq() iter.Seq[string] {
	return func(yield func(string) bool) {
		// Open the file. Return immediately on error.
		f, err := os.Open(r.Path)
		if err != nil {
			return
		}
		// Defer ensures the file closes when the closure returns.
		// This handles both normal completion and early break.
		defer f.Close()

		scanner := bufio.NewScanner(f)
		for scanner.Scan() {
			// Yield the line. Stop if the consumer breaks.
			if !yield(scanner.Text()) {
				return
			}
		}
	}
}

func main() {
	// Create a temporary file for demonstration.
	f, _ := os.CreateTemp("", "example*.txt")
	f.WriteString("line one\nline two\nline three\n")
	f.Close()

	// Iterate over lines. Break after the second line.
	count := 0
	for line := range FileLines{Path: f.Name()}.Seq() {
		fmt.Println(line)
		count++
		if count == 2 {
			break
		}
	}
}

The defer f.Close() inside the closure is the key pattern. The closure returns when the iterator finishes or when yield returns false. defer runs on return. This guarantees the file closes even if the consumer breaks early.

The receiver name is r, a single letter matching the type FileLines. Go convention favors short receiver names. Use (r *Reader) or (s *Slice), not (this *Reader) or (self *Reader).

Iterators don't carry context. If your iteration does I/O, pass context.Context as the first argument to the iterator function. The iterator itself is just data flow. Context is plumbing. Run it through every long-lived call site.

Pitfalls and errors

The most common bug is ignoring the yield return value. If the consumer breaks, yield returns false. If you don't check it, your iterator keeps running. It does work nobody wants. It might hold locks or file handles open longer than necessary. The compiler won't stop you here. It's a logic error. Always write if !yield(v) { return }.

Loop variable capture is another trap. If you use a loop variable inside a closure, you must be careful about scoping. Go 1.22 changed loop variable scoping to create a new variable per iteration, which fixes the classic bug. However, if you write code that relies on the old behavior or mixes patterns, the compiler guards you.

If you try to capture a loop variable in a way that conflicts with the new rules, the compiler rejects the program with loop variable captured by func literal. This error message appears when the compiler detects a capture that could lead to confusion. Go 1.22+ handles most cases automatically, but the error remains for edge cases.

Iterators are not safe for concurrent use. iter.Seq assumes a single consumer. If you pass the same iterator to multiple goroutines, you get data races. Use channels when you need concurrent producers or consumers.

The worst iterator bug is the one that never logs. If your iterator swallows errors or leaks resources silently, you won't know until the system degrades. Check errors. Defer cleanup. Respect the break signal.

When to use iterators

Pick the right tool for the job. Iterators shine in specific scenarios.

Use a slice when you need random access, length, or the data fits comfortably in memory. Slices are fast for indexing and sorting.

Use iter.Seq when you want to chain transformations without allocating intermediate slices. Iterators keep memory usage constant and allow early termination.

Use iter.Pull when you need to convert an iterator to a callback-based consumer pattern or integrate with older code that expects a next function.

Use a channel when multiple goroutines need to produce or consume values concurrently. Channels handle synchronization and buffering.

Use a simple for loop when the logic is short and you don't need to pass the iteration as a value. Don't add abstraction overhead for trivial cases.

Trust the type system. Wrap the value or change the design.

Where to go next

A custom iterator is a function that produces a stream of values on demand instead of creating a full list in memory. It works like a generator that yields one item at a time when you ask for it. This saves memory and lets you process large datasets efficiently.