How to Use benchstat to Compare Benchmark Results

Use benchstat to statistically compare benchmark results by piping output from multiple go test runs into the tool. First, run your benchmarks multiple times to generate result files, then pass those files to benchstat for analysis.

go test -bench=. -benchmem -count=10 > old.txt
go test -bench=. -benchmem -count=10 > new.txt
benchstat old.txt new.txt

This command runs the benchmarks 10 times each, saves the output to text files, and benchstat calculates the mean, standard deviation, and percentage change to determine if the performance difference is statistically significant.

benchstat is a tool that helps you decide if a code change actually made your program faster or slower. It takes the raw timing data from running your tests multiple times and uses statistics to filter out random noise. Think of it like a referee that tells you if a runner truly improved their time or if they just got lucky on one specific day.