The loop that eats memory
You are writing a function to generate a SQL INSERT statement. You loop over a list of values, appending each one to a string variable. The function works for ten rows. You test with ten thousand rows. The CPU spikes. Memory usage climbs. The garbage collector runs constantly. You just hit the quadratic trap.
String concatenation in Go looks like basic arithmetic, but the underlying mechanics dictate performance. Go strings are immutable. Every time you combine strings, the runtime allocates new memory and copies data. The right tool depends on whether you have two strings, a slice of strings, or a loop that grows the result. Picking the wrong tool costs cycles and memory.
Strings are immutable labels
Go strings are immutable. A string is a header containing a pointer to a byte array and a length. The byte array cannot be modified. If you try to change a character, the compiler rejects the program. Concatenation creates a new byte array. It copies the contents of the operands into the new array. The original strings remain unchanged.
This design makes strings safe to share. You can pass a string to a goroutine without worrying about races. The cost is allocation. Allocating memory is cheap for small strings. It becomes expensive when you allocate repeatedly in a loop.
When you concatenate inside a loop with the + operator, the compiler cannot predict the final size. It allocates a buffer, copies the current string, copies the new fragment, and discards the old buffer. The next iteration repeats the process. The total work grows quadratically with the number of iterations. For a thousand iterations, you copy the data thousands of times. The runtime spends more time moving bytes than doing useful work.
Strings are immutable. Every concatenation is a new allocation.
Simple concatenation with +
The + operator joins strings. It works for a fixed number of operands. The compiler often optimizes simple chains into a single allocation. Use + for quick joins where the operands are known at the call site.
Here's the simplest way to glue strings together using the plus operator.
package main
import "fmt"
func main() {
// + creates a new string by copying both operands
greeting := "Hello" + " " + "World"
fmt.Println(greeting)
}
The compiler sees the literals and the operator. It calculates the total length. It allocates a single buffer. It copies the bytes. The result is efficient. The optimization applies to constants and variables as long as the chain is short and static. The moment you move + into a loop, the optimization disappears.
Joining slices with strings.Join
When you have a slice of strings, strings.Join is the standard tool. It takes a slice and a separator. It returns a single string with the separator between elements. strings.Join avoids the quadratic trap by measuring the total size before allocating.
Here's how to combine a slice of strings with a separator.
package main
import (
"fmt"
"strings"
)
func main() {
// Join calculates total size in one pass
parts := []string{"user", "admin", "guest"}
// Separator is inserted between elements, not at ends
result := strings.Join(parts, ", ")
fmt.Println(result)
}
strings.Join performs two passes over the slice. The first pass sums the lengths of all strings and adds the length of the separator multiplied by the number of gaps. The second pass allocates a buffer of the exact size and copies the data. This approach guarantees a single allocation. The complexity is linear relative to the total length.
If the slice is empty, strings.Join returns an empty string. If the slice has one element, it returns that element without the separator. The function handles edge cases correctly.
Join measures before it allocates. Never guess the size.
Building incrementally with strings.Builder
When you need to build a string in a loop, strings.Builder is the correct choice. It maintains an internal byte slice that grows as you write. It reuses the buffer across writes. It avoids repeated allocations.
Here's how to build a large string efficiently using strings.Builder.
package main
import (
"fmt"
"strings"
)
func buildLog(entries []string) string {
// Builder reuses the same underlying buffer
var sb strings.Builder
// Pre-allocate capacity to avoid resizing during writes
sb.Grow(len(entries) * 10)
for _, entry := range entries {
// WriteString appends directly to the buffer
sb.WriteString(entry)
sb.WriteString("\n")
}
// String() returns the final immutable string
return sb.String()
}
func main() {
logs := []string{"error: disk full", "warn: cpu high"}
fmt.Println(buildLog(logs))
}
strings.Builder starts with a small buffer. When you call WriteString, it appends to the buffer. If the buffer is full, it allocates a larger buffer and copies the data. This resizing is amortized, so the average cost per write is low. Calling Grow with an estimated size prevents resizing entirely. The String method returns the buffer contents as a string.
After calling String, the builder should not be reused. The documentation states the builder is invalid after String returns. In practice, the implementation often allows reuse, but the contract forbids it. Create a new builder or call Reset if you need to build another string. Reset clears the buffer but keeps the memory, allowing reuse without reallocation.
strings.Builder is not thread-safe. If two goroutines write to the same builder, you get a data race. The race detector flags this immediately. Always use one builder per goroutine.
Builder reuses the buffer. Reset it, don't recreate it.
Formatting with fmt.Sprintf
Sometimes you need to mix types. fmt.Sprintf formats values into a string. It handles integers, floats, and custom formatters. It is convenient for one-off messages. It is slower than + or Builder.
The fmt package parses the format string every time you call Sprintf. This parsing adds overhead. In a tight loop, Sprintf accumulates cost. Use Sprintf for logging or user-facing messages where readability matters more than performance. Avoid it in hot paths.
fmt is type-unsafe regarding format verbs. The compiler checks the number of arguments, but not the types. If you pass a string where %d expects an integer, the output is wrong, not an error. fmt.Sprintf("%d", "hello") prints 0. This is a runtime logic error. The compiler does not catch it. Be careful with format strings.
fmt is convenient. It is not fast.
Pitfalls and compiler errors
Common mistakes include using + in a loop, misusing strings.Builder, or ignoring type errors.
You cannot add a number to a string with +. The compiler rejects this with invalid operation: operator + not defined on int. You must convert the number to a string first. Use strconv.Itoa for integers or fmt.Sprintf for formatted numbers.
If you try to modify a string byte, the compiler rejects the program with cannot assign to string index. Strings are immutable. Use a byte slice if you need mutation.
strings.Builder is not safe for concurrent use. If you share a builder across goroutines, the race detector reports a data race. The output may be corrupted. Use a builder per goroutine or synchronize access with a mutex.
The + operator in a loop causes quadratic performance. The compiler does not warn about this. The code compiles and runs. It just gets slow. Profile your code to find these bottlenecks.
Don't pass a *string. Strings are cheap to pass by value. The header is two words. Passing a pointer adds indirection without saving memory. Pass the string directly.
Decision matrix
Use the + operator when you are joining a small, fixed number of strings known at the call site. Use strings.Join when you have a slice of strings and need a separator between them. Use fmt.Sprintf when you need to format values like integers or floats into a string and performance is not critical. Use strings.Builder when you are appending strings in a loop or building a large result incrementally. Use bytes.Buffer when you need to read from the buffer as well, or when you need to satisfy an io.ReadWriter interface.
Pick the tool that matches the shape of your data.