When the binary gets too heavy
You are building a command-line tool that needs a massive lookup table, or a web server that serves static assets. You want a single binary that runs anywhere without missing files. You add //go:embed assets/ to your code. It works perfectly for your CSS, JavaScript, and small JSON configs. Then you try to embed a 3GB database dump. The build fails. Or you try to embed a file from a shared configuration directory outside your project, and the compiler yells. Or you are on Windows and a backslash in the path breaks the pattern parser.
go:embed is a powerful feature, but it is not a magic wand for every file on your disk. It has hard boundaries enforced by the compiler and the module system. When you hit those boundaries, you need to shift from compile-time embedding to runtime loading or archiving strategies.
How go:embed works under the hood
The //go:embed directive is a compiler instruction. It tells the Go compiler to read files from the filesystem and bake them directly into the binary during the build process. The result is a Go variable that holds the file contents.
When you embed a single file into a []byte, the compiler generates a literal array of bytes inside the generated code. When you embed a directory into an embed.FS, the compiler creates a structure that implements the io/fs.FS interface. This structure contains the file tree and the data, allowing you to use standard library functions like http.FileServer or template.ParseFS to serve or parse the assets.
Think of go:embed like baking cookies into a cake. The cookies are inside the cake. You can eat the cake anywhere without needing a separate jar of cookies. But you cannot bake a cookie that is bigger than the oven, and you cannot add a cookie after the cake is already baked. The limits are physical constraints of the baking process.
The hard limits
The embed package enforces three major constraints. These are not bugs; they are design choices to protect the compiler, the binary size, and build reproducibility.
The 2GB file size cap
You cannot embed a single file larger than 2GB. The embed package checks the size of every file before processing. If a file exceeds the limit, the build fails.
This limit exists because the compiler generates Go source code to represent the embedded data. Generating a []byte literal for a multi-gigabyte file would create an intermediate representation that could crash the compiler or produce a binary that is impossible to link. The 2GB cap is a safety valve. It prevents the compiler from running out of memory and stops developers from accidentally creating unmanageable binaries.
Backslashes in file names and paths
The //go:embed directive uses glob patterns to match files. The pattern parser expects forward slashes as path separators. If a file name or path contains a backslash, the parser may interpret it as an escape character or reject it as invalid syntax.
This is a common trap on Windows, where file paths naturally use backslashes. The directive requires forward slashes regardless of the operating system. If you try to embed a file with a backslash in the name, the compiler rejects the program with an error like embed: pattern ...: invalid syntax. The fix is to rename the file or use a path with forward slashes.
Files outside the module directory
You cannot embed files that live outside the Go module root. The module system defines the boundary of your project. go:embed respects this boundary strictly.
This restriction guarantees reproducible builds. If go:embed allowed arbitrary paths, moving the project directory or changing the working directory could silently change which files get embedded. By locking embedding to the module root, the compiler ensures that the same source code always produces the same binary, regardless of where you run the build. Attempting to embed a file outside the module triggers an error such as embed: open ...: file does not exist or a module boundary violation.
Minimal example: Hitting the wall
Here is a simple program that tries to embed a file. If the file is too large, has a bad name, or is in the wrong place, the compiler stops the build.
package main
import (
"embed"
"fmt"
)
//go:embed data.bin
var data []byte
//go:embed assets/
var assetFS embed.FS
func main() {
// Print the size of the embedded data.
fmt.Println("Data size:", len(data))
// List files in the embedded filesystem.
entries, err := assetFS.ReadDir("assets")
if err != nil {
panic(err)
}
for _, e := range entries {
fmt.Println("Asset:", e.Name())
}
}
If data.bin is 3GB, the compiler rejects the build with an error indicating the file exceeds the size limit. The build stops before the binary is produced. You cannot work around this by changing flags or settings. The limit is hard-coded in the embed package.
If assets/ contains a file named config\settings.json, the compiler may fail with a pattern syntax error. The backslash breaks the glob parser.
Convention aside: The //go:embed comment must appear immediately above the variable declaration. No blank lines are allowed between the comment and the variable. gofmt preserves the directive comment, but it does not fix syntax errors in the pattern. If the comment is separated by a blank line, the compiler ignores it, and the variable remains empty.
Realistic workaround: Runtime loading and archives
When you hit the limits, you move the file handling from compile time to runtime. The binary stays small, and the files are loaded when the program starts. This shifts the complexity from the build process to the deployment process.
Loading large files from disk
For files larger than 2GB, read them from the filesystem at runtime. This requires the file to be present on the disk where the program runs.
package main
import (
"fmt"
"io"
"os"
)
// LoadAsset reads a file from the filesystem at runtime.
// This bypasses compile-time size limits but requires the file to exist when the program runs.
func LoadAsset(path string) ([]byte, error) {
f, err := os.Open(path)
if err != nil {
return nil, err
}
defer f.Close()
// ReadAll loads the entire file into memory.
// For multi-gigabyte files, consider streaming the data instead of buffering it all at once.
return io.ReadAll(f)
}
func main() {
data, err := LoadAsset("large-data.bin")
if err != nil {
fmt.Println("Failed to load asset:", err)
return
}
fmt.Println("Loaded", len(data), "bytes")
}
This approach works for any file size, limited only by available RAM and disk space. The trade-off is that the deployment must ensure the file is copied to the correct location. You lose the single-binary convenience.
Bundling with archives
When you have many large files, bundling them into a zip or tar archive is often cleaner than managing a directory of loose files. You can distribute the archive alongside the binary and extract it on first run.
package main
import (
"archive/zip"
"fmt"
"io"
"os"
)
// ExtractZip reads a zip archive from disk and writes its contents to a destination directory.
// This allows bundling large assets separately from the binary and extracting them at startup.
func ExtractZip(zipPath, destDir string) error {
r, err := zip.OpenReader(zipPath)
if err != nil {
return err
}
defer r.Close()
for _, f := range r.File {
// Skip directories for simplicity in this example.
if f.FileInfo().IsDir() {
continue
}
rc, err := f.Open()
if err != nil {
return err
}
defer rc.Close()
// Create the output file on disk.
outFile, err := os.Create(destDir + "/" + f.Name)
if err != nil {
return err
}
defer outFile.Close()
// Copy the file content from the archive to the disk.
_, err = io.Copy(outFile, rc)
if err != nil {
return err
}
}
return nil
}
func main() {
err := ExtractZip("assets.zip", "./extracted")
if err != nil {
fmt.Println("Extraction failed:", err)
return
}
fmt.Println("Assets extracted successfully")
}
Archives compress the data, reducing disk usage and transfer time. The extraction adds a small CPU cost at startup, but it keeps the deployment package tidy. You can embed a small manifest file using go:embed to track the version of the external archive, while keeping the heavy data outside the binary.
Convention aside: Functions that take a context.Context should accept it as the first parameter, conventionally named ctx. If your extraction or loading logic is long-running, pass a context so you can cancel the operation if the user interrupts the program.
Pitfalls and errors
Runtime loading introduces new failure modes. The compiler can no longer check if files exist. Errors happen when the program runs.
Missing files at runtime
If you switch from go:embed to runtime loading, the program will panic or return an error if the file is missing. The compiler error pattern ...: no matching files found is replaced by a runtime error like open data.bin: no such file or directory.
Always check errors when opening files. Do not assume the file exists just because it existed during development. The deployment environment might have different paths or permissions.
Memory exhaustion
Loading a large file with io.ReadAll allocates a slice in memory. If the file is larger than the available RAM, the program crashes with an out-of-memory panic. For very large files, stream the data using io.Copy or read it in chunks. Do not buffer multi-gigabyte files into memory unless you have a specific reason to hold the entire content at once.
Path separators on Windows
Runtime file paths also need care on Windows. os.Open handles backslashes correctly, but if you construct paths by concatenating strings, use filepath.Join instead of manual string building. filepath.Join uses the correct separator for the operating system. Hard-coding forward slashes in path concatenation can cause issues on Windows if the underlying system call expects backslashes.
Goroutine leaks with channels
If you use goroutines to load assets in parallel, ensure you have a cancellation path. Goroutine leaks happen when a goroutine waits on a channel that never gets closed. Always use context.Context to signal cancellation, and close channels when the sender is done. The worst goroutine bug is the one that never logs.
Decision: when to use what
Choose the strategy that matches your asset size, deployment constraints, and update frequency.
Use go:embed when you have small assets like templates, icons, or config files that must travel with the binary and fit within the 2GB limit.
Use runtime file loading when assets exceed 2GB or need to be updated without rebuilding the binary.
Use archive/zip or archive/tar when you need to bundle many large files together and extract them on first run to keep the deployment package compact.
Use a sidecar directory structure when your deployment environment already provides the assets alongside the binary and you want zero extraction overhead.
Use go:embed for the manifest or metadata files that describe the external assets, even when the assets themselves are loaded at runtime.
Embed limits are hard. Work around them at runtime. The compiler protects you from bloated binaries. Respect the 2GB wall and keep your deployment strategy explicit.