The slow walk and the fast walk
You are building a tool to scan a project for configuration files. You write a recursive function that calls os.Stat on every path. It works. It also takes forever on a large directory. The CPU spends more time asking the operating system "what is this?" than actually processing the files. Go provides two ways to walk a directory tree. One is the legacy approach that calls os.Stat on every single entry. The other is the modern approach that fetches directory metadata in bulk. The performance difference is not marginal. It can be orders of magnitude.
The catalog versus the spine
Think of walking a directory like checking a library. filepath.Walk is like walking to every bookshelf, picking up every book, checking the spine to see if it's a novel or a textbook, and then putting it back. You do this for every single item. filepath.WalkDir is like asking the librarian for a catalog of the whole shelf at once. The catalog tells you the type and size of every item without touching the books. You only pick up the books when you actually need to read them.
The catalog is fs.DirEntry. The book inspection is os.Stat. filepath.Walk calls os.Stat on every entry. filepath.WalkDir calls os.ReadDir once per directory and returns fs.DirEntry objects. The DirEntry interface gives you the name, type, and whether it is a directory without a syscall. You only pay the cost of a syscall when you call d.Info() to get details like size or modification time.
The minimal walk
Here is the simplest way to walk a directory with the modern API. The code prints the path of every regular file.
package main
import (
"fmt"
"io/fs"
"path/filepath"
)
// WalkExample prints paths of all regular files in the current directory.
func WalkExample() error {
// WalkDir descends the tree and calls the callback for each entry.
return filepath.WalkDir(".", func(path string, d fs.DirEntry, err error) error {
// The err argument captures errors from reading the directory.
if err != nil {
return err
}
// IsDir checks the file mode without a syscall.
if d.IsDir() {
return nil
}
// Print the path for regular files.
fmt.Println(path)
return nil
})
}
The callback receives three arguments. The path is the full path to the entry. The d is the fs.DirEntry with metadata. The err is any error encountered while reading that entry. If the callback returns an error, the walk stops immediately. If you return nil, the walk continues. The if err != nil check inside the callback is the standard Go error handling pattern. It looks repetitive, but it forces you to decide what to do with every error. You can return the error to stop, return nil to ignore, or return a sentinel error to change behavior.
How the walk executes
When you call filepath.WalkDir, the function reads the root directory using os.ReadDir. This returns a slice of fs.DirEntry objects. The walk function calls your callback for each entry. If the entry is a directory, the walk descends into it and repeats the process. The traversal is depth-first.
The callback can influence the traversal by returning specific errors. Returning fs.SkipDir tells the walk to skip the current directory and move on to the next sibling. Returning fs.SkipAll stops the walk entirely. These are sentinel errors defined in the io/fs package. They implement the error interface, so the compiler accepts them as return values.
If you try to return fs.SkipDir from a function that expects a different error type, the compiler rejects the program with a type mismatch error. The callback signature requires error, and fs.SkipDir is an error, so this works correctly. If you shadow the fs import, you might get undefined: fs from the compiler. The convention is to import io/fs and use the fs prefix for types like DirEntry and SkipDir.
A realistic filter
Here is a realistic example that collects log files while skipping hidden directories. This pattern appears in backup tools, linters, and search utilities.
package main
import (
"fmt"
"io/fs"
"path/filepath"
"strings"
)
// FindLogFiles returns paths of all .log files, skipping hidden directories.
func FindLogFiles(root string) ([]string, error) {
var results []string
// WalkDir traverses the tree starting at root.
err := filepath.WalkDir(root, func(path string, d fs.DirEntry, err error) error {
// Propagate directory read errors to stop the walk.
if err != nil {
return err
}
// Skip hidden directories to avoid .git or .cache bloat.
if d.IsDir() && strings.HasPrefix(d.Name(), ".") {
return fs.SkipDir
}
// Match .log extension on non-directory entries.
if !d.IsDir() && strings.HasSuffix(d.Name(), ".log") {
results = append(results, path)
}
return nil
})
// Return the accumulated list or the error from traversal.
return results, err
}
The code checks d.IsDir() to distinguish directories from files. It uses strings.HasPrefix to detect hidden directories. Returning fs.SkipDir prevents the walk from descending into .git or .cache. The filter checks !d.IsDir() before checking the extension. This avoids calling HasSuffix on directories. The results are collected in a slice. The function returns the slice and any error from the walk.
Performance and syscalls
The speed difference comes from system calls. Every call to os.Stat triggers a system call. The process pauses, the kernel loads, the disk is queried, and the result is marshaled back. Doing this for 10,000 files means 10,000 context switches. WalkDir uses os.ReadDir, which asks the kernel for the whole directory at once. The kernel returns a batch of entries. The number of syscalls drops from N to D, where D is the number of directories.
If you have a directory with 10,000 files and 10 subdirectories, filepath.Walk performs roughly 10,010 syscalls. filepath.WalkDir performs roughly 11 syscalls. The difference is dramatic on slow storage or network mounts. The fs.DirEntry type was added in Go 1.16 specifically to address this performance gap. New code should always use WalkDir.
Handling metadata lazily
The DirEntry interface includes an Info() method. This method calls os.Lstat internally. Calling Info() on every entry defeats the purpose of WalkDir. Only call Info() when you need the size, modification time, or permissions. If you just need the name or type, use Name() and IsDir().
The Type() method returns a fs.FileMode. This value contains bits for regular file, directory, symlink, and permissions. You can check d.Type().IsDir() or d.Type()&os.ModeSymlink != 0. The IsDir() method is just a wrapper around Type().IsDir(). Using Type() directly can be slightly faster if you need to check multiple flags, but IsDir() is clearer for simple checks. The convention is to use IsDir() unless you need the mode bits for other reasons.
Pitfalls and errors
A common mistake is returning a standard error when you want to skip a directory. If you return a standard error, the walk halts entirely. Use fs.SkipDir to skip the current directory and continue with siblings. Use fs.SkipAll to stop the walk completely.
Another pitfall is symlinks. WalkDir does not follow symbolic links. If you pass a symlink to a directory as the root, it walks the symlink target. If you encounter a symlink inside the tree, it reports it as a file entry. The callback receives the symlink entry, not the target. If you need to resolve symlinks, you must call filepath.EvalSymlinks manually on the path. The Type() method returns a mode with the ModeSymlink bit set for symlinks. You can check this bit to detect symlinks.
If you try to walk a path that doesn't exist, WalkDir returns an error immediately. The error message looks like lstat /path: no such file or directory. If you forget to handle the error from WalkDir, the compiler warns err declared and not used. The convention is to check the error and return it or handle it.
If you pass a non-directory to WalkDir, it returns an error like not a directory. The walk function expects a directory root. If you need to walk a single file, handle it separately.
When to use what
Use filepath.WalkDir when you need to traverse a directory tree efficiently. It reads directory entries in bulk and avoids the overhead of calling os.Stat on every file. This is the default choice for almost all directory walking tasks.
Use filepath.Walk only when you are maintaining legacy code that depends on os.FileInfo. It calls os.Lstat on every entry, which makes it significantly slower for large trees. The performance difference can be orders of magnitude on directories with thousands of files.
Use os.ReadDir with manual recursion when you need fine-grained control over the traversal order or want to implement a custom queue-based walker. This approach gives you full control but requires more boilerplate to handle errors and directory descent.
Use filepath.Glob when you need to match a simple pattern like *.go in a single directory. It does not recurse into subdirectories. It returns a list of matching paths without walking the tree.