When SQL feels too heavy
You're building a CLI tool that needs to remember the last sync timestamp. Or a microservice that caches user preferences locally. You don't need a full SQL database with joins, migrations, and a separate server process. You don't want to spin up Redis just to store a few megabytes of configuration. You need a file on disk that behaves like a map, with transactions and durability. That's where bbolt comes in.
What bbolt actually is
bbolt is an embedded key-value store. "Embedded" means the library runs inside your Go process. There is no localhost:5432. Your program opens the database file, reads the pages, and writes the pages directly. It uses a B-tree structure, which keeps lookups fast even as the file grows.
Data lives in buckets. A bucket is like a namespace or a folder. You cannot store keys at the root of the database; every key must belong to a bucket. This design keeps the B-tree organized and allows you to group related data. Keys and values are raw byte slices. bbolt doesn't care about strings, integers, or structs. It stores bytes. You handle serialization. This gives you total control: store JSON, Protocol Buffers, binary data, or plain text.
The minimal write and read
Here's the bare minimum to write and read a value. The code opens the database, creates a bucket, stores a key, and reads it back.
package main
import (
"fmt"
"log"
"go.etcd.io/bbolt"
)
func main() {
// Open creates the file if missing, or opens existing.
// 0600 sets permissions to read/write for owner only.
// nil passes default options; production code usually tunes timeouts.
db, err := bbolt.Open("app.db", 0600, nil)
if err != nil {
log.Fatal(err)
}
// Close releases the file lock and flushes buffers.
// Defer ensures cleanup even if main panics.
defer db.Close()
// Update starts a write transaction.
// The function runs atomically; any error rolls back changes.
err = db.Update(func(tx *bbolt.Tx) error {
// CreateBucketIfNotExists makes the bucket once.
// Buckets are required containers for keys.
b, err := tx.CreateBucketIfNotExists([]byte("settings"))
if err != nil {
return err
}
// Put writes the key-value pair.
// Keys and values are byte slices; strings must be converted.
return b.Put([]byte("theme"), []byte("dark"))
})
if err != nil {
log.Fatal(err)
}
// View starts a read-only transaction.
// Multiple views can run concurrently without blocking each other.
err = db.View(func(tx *bbolt.Tx) error {
b := tx.Bucket([]byte("settings"))
// Get returns nil if the key doesn't exist.
val := b.Get([]byte("theme"))
fmt.Printf("Theme: %s\n", string(val))
return nil
})
}
How transactions and buckets work
Transactions are mandatory in bbolt. There is no auto-commit. Every read or write happens inside a transaction function passed to db.View or db.Update.
db.Update acquires an exclusive lock. Only one write transaction can run at a time. If your update function returns an error, bbolt rolls back the changes. If it returns nil, bbolt commits the transaction, writing the changes to the database file. This atomicity protects your data from partial writes.
db.View acquires a shared lock. Multiple read transactions can run concurrently. They see a snapshot of the database as it existed when the transaction started. Long-running read transactions block writes, so keep views short.
Buckets are created inside transactions. tx.CreateBucket fails if the bucket already exists. tx.CreateBucketIfNotExists is safer for initialization code. Once a bucket exists, you retrieve it with tx.Bucket. If the bucket doesn't exist, tx.Bucket returns nil. Always check for nil before calling Get or Put, or the program panics.
The compiler enforces byte slices strictly. If you pass a string variable to Put, the compiler rejects the program with cannot use s (variable of type string) as []byte value in argument. You must convert explicitly: []byte(s). This prevents accidental encoding issues and reminds you that bbolt stores raw bytes.
Buckets are the gatekeepers. Hold transactions short, or block the world.
Persisting structs and iterating
Real code usually involves structs. Here's a pattern for persisting a configuration object and iterating over keys.
package main
import (
"encoding/json"
"fmt"
"log"
"go.etcd.io/bbolt"
)
// State holds application configuration.
// Public fields allow encoding; private fields stay internal.
type State struct {
Version int `json:"version"`
Token string `json:"token"`
}
// SaveState persists the state to the database.
// It uses a write transaction to ensure atomicity.
func SaveState(db *bbolt.DB, s State) error {
return db.Update(func(tx *bbolt.Tx) error {
b, err := tx.CreateBucketIfNotExists([]byte("state"))
if err != nil {
return err
}
// Marshal converts the struct to JSON bytes.
// JSON is a safe format for key-value storage.
data, err := json.Marshal(s)
if err != nil {
return err
}
// Store under a fixed key.
// Overwriting the same key updates the value atomically.
return b.Put([]byte("current"), data)
})
}
// LoadState retrieves the state from the database.
// It returns a zero-value State if the key is missing.
func LoadState(db *bbolt.DB) (State, error) {
var s State
err := db.View(func(tx *bbolt.Tx) error {
b := tx.Bucket([]byte("state"))
if b == nil {
return nil
}
data := b.Get([]byte("current"))
if data == nil {
return nil
}
// Unmarshal parses JSON back into the struct.
// Errors here indicate corruption or format mismatch.
return json.Unmarshal(data, &s)
})
return s, err
}
// ListKeys prints all keys in the settings bucket.
// ForEach calls the function for every key-value pair.
func ListKeys(db *bbolt.DB) error {
return db.View(func(tx *bbolt.Tx) error {
b := tx.Bucket([]byte("settings"))
if b == nil {
return nil
}
// ForEach iterates in sorted key order.
// Returning an error stops the iteration immediately.
return b.ForEach(func(k, v []byte) error {
fmt.Printf("Key: %s, Value: %s\n", string(k), string(v))
return nil
})
})
}
ForEach iterates over all keys in a bucket in sorted order. It's efficient for small to medium buckets. For large datasets, use a Cursor to iterate in chunks or seek to a specific range. ForEach stops early if the callback returns an error, which is useful for cancellation or early exit.
bbolt transactions don't accept context.Context. If you need cancellation, manage it outside the transaction. Return early from the transaction function if the context is done. This keeps the transaction lifecycle explicit and prevents hidden dependencies.
Pitfalls and compiler traps
BoltDB has a few traps that catch developers off guard.
File locking is strict. If you open the database with write mode, the file is locked. Opening it again with write mode fails with database is locked. Always close the database handle when finished. Use defer db.Close() immediately after opening.
Transactions must be committed or rolled back. If you hold a transaction open for too long, you block other writers. In a web server, never start a transaction in the handler and return it to the caller. Start and finish the transaction within the handler. Long transactions also prevent the database from compacting, which can cause the file to grow indefinitely.
Byte slices are raw. The compiler won't stop you from storing a string, but it won't help you retrieve it. If you store []byte("hello") and retrieve it, you get a byte slice. You must convert back to string with string(val). Forgetting this leads to type errors or garbled output.
Buckets must exist. Calling tx.Bucket on a non-existent bucket returns nil. Calling Get or Put on a nil bucket panics. Always check for nil or use CreateBucketIfNotExists before accessing data.
The compiler enforces imports. If you import bbolt but don't use it, you get imported and not used. If you use bbolt but forget the import, you get undefined: bbolt. Keep imports clean.
Transactions are the gatekeepers. Check for nil buckets, convert bytes explicitly, and close handles immediately.
Choosing the right storage
Use bbolt when you need a simple, embedded key-value store with durability and no external dependencies. Use bbolt when you are building a CLI tool or a microservice that requires local state without managing a database server. Use bbolt when you need fast reads and writes for small to medium datasets that fit in memory. Use database/sql when you need complex queries, joins, or schema migrations across multiple tables. Use an in-memory map when the data can be rebuilt on restart and you need maximum read/write speed. Use a dedicated cache like Redis when multiple processes or machines need to share the same data.