How to Use hashicorp/go-plugin for Plugin Systems

Core
2026-04-17T16:32:01+00:00
Use `hashicorp/go-plugin` to create a plugin system by defining a shared interface, implementing it in a plugin binary, and using the `plugin.Serve` and `plugin.Client` functions to handle the gRPC handshake and protocol negotiation.

The plugin problem: bloat versus isolation

You built a CLI tool that processes data. Users love it, but they want custom output formats. You could add every format to the core, but the codebase bloats and every change requires a full release. Or you could ask users to fork the repo, which creates a maintenance nightmare where everyone drifts apart. You need a way to load external code at runtime, safely, without crashing the main process if the plugin misbehaves.

hashicorp/go-plugin solves this by turning your Go application into a host that spawns and communicates with separate plugin binaries. The library handles process management, protocol negotiation, and serialization. You define a shared interface, and the library generates the remote procedure call layer automatically. Your host code calls plugin methods as if they were local functions, while the plugin runs in its own process.

The plugin is a separate process. Treat it like a remote service, not a local function.

Concept: The USB standard for Go

Think of go-plugin like a USB standard. Your application is the computer. The plugin is a USB device. Both sides agree on the USB protocol. The computer does not need to know how the device stores data; it just sends commands and gets responses. If the device fails, the computer stays running.

go-plugin implements this pattern using gRPC under the hood, but it hides the complexity. You do not write .proto files. You do not manually serialize structs. You define a Go interface, and the library uses reflection to generate the gRPC service and client stubs at runtime. The host and plugin exchange a handshake over standard output to negotiate the connection, then switch to gRPC for all subsequent communication.

This design gives you process isolation. If the plugin panics, the host receives a gRPC error instead of crashing. You can also update the plugin without restarting the host, and you can distribute plugins as standalone binaries that users compile or download independently.

The library abstracts gRPC completely. You write Go interfaces, and go-plugin handles the rest.

The plugin is a separate process. Treat it like a remote service, not a local function.

Minimal example: Shared interface, plugin, and host

Start with a shared package that both the host and plugin import. This package defines the contract. Both sides must use the exact same version of this package to ensure the interface definitions match.

Here is the shared interface. It defines the methods the plugin must implement.

// shared/plugin.go
package shared

import "context"

// Greeter defines the contract between host and plugin.
type Greeter interface {
    // Greet returns a personalized message.
    // Context first follows Go convention for cancellable operations.
    Greet(ctx context.Context, name string) (string, error)
}

The plugin binary implements this interface and registers itself with the plugin system. The Serve function starts the gRPC server and waits for the host to connect.

Here is the plugin implementation. The receiver name g follows Go convention: one or two letters matching the type.

// plugin/impl.go
package main

import "context"

// greeterImpl implements the shared.Greeter interface.
type greeterImpl struct{}

func (g *greeterImpl) Greet(ctx context.Context, name string) (string, error) {
    // Receiver name g matches the type initial.
    // Context first follows convention for cancellable calls.
    return "Hello, " + name + " from plugin!", nil
}

Here is the plugin entry point. It configures the handshake and serves the gRPC plugin.

// plugin/main.go
package main

import (
    "github.com/hashicorp/go-plugin"
)

func main() {
    // Serve starts the gRPC server and waits for the host.
    plugin.Serve(&plugin.ServeConfig{
        // HandshakeConfig validates the host before connecting.
        HandshakeConfig: plugin.HandshakeConfig{
            ProtocolVersion:  1,
            MagicCookieKey:   "BASIC_PLUGIN",
            MagicCookieValue: "handshake",
        },
        // Plugins maps names to plugin implementations.
        Plugins: map[string]plugin.Plugin{
            "greeter": &plugin.GRPCPlugin{
                // Server provides the concrete implementation.
                Server: func(broker *plugin.GRPCBroker) interface{} {
                    return &greeterImpl{}
                },
            },
        },
    })
}

The host binary spawns the plugin process and retrieves the client stub. The host must use the same handshake configuration and import the same shared package.

Here are the configuration variables. Extracting them keeps the client config concise.

// host/vars.go
package main

import (
    "github.com/hashicorp/go-plugin"
    "example.com/myapp/shared"
)

// handshake defines the protocol negotiation parameters.
var handshake = plugin.HandshakeConfig{
    ProtocolVersion:  1,
    MagicCookieKey:   "BASIC_PLUGIN",
    MagicCookieValue: "handshake",
}

// plugins maps names to plugin implementations.
var plugins = map[string]plugin.Plugin{
    "greeter": &plugin.GRPCPlugin{
        // Client returns the wrapper for remote calls.
        Client: func(broker *plugin.GRPCBroker, c *plugin.Client) interface{} {
            return &shared.GreeterClient{Client: c, Broker: broker}
        },
    },
}

Here is the host function that creates the client.

// host/config.go
package main

import (
    "os/exec"
    "github.com/hashicorp/go-plugin"
)

// newPluginClient returns a configured plugin client.
func newPluginClient() *plugin.Client {
    return plugin.NewClient(&plugin.ClientConfig{
        // HandshakeConfig must match the plugin exactly.
        HandshakeConfig: handshake,
        // Plugins registers the client stubs.
        Plugins: plugins,
        // Cmd specifies the plugin binary to execute.
        Cmd: exec.Command("./plugin"),
    })
}

Here is the main function that connects and calls the plugin.

// host/main.go
package main

import (
    "context"
    "log"
    "github.com/hashicorp/go-plugin"
    "example.com/myapp/shared"
)

func main() {
    client := newPluginClient()

    // Client() starts the process and negotiates the handshake.
    rpcClient, err := client.Client()
    if err != nil {
        log.Fatal(err)
    }
    // Kill() terminates the plugin process on exit.
    defer client.Kill()

    // Dispense retrieves the plugin instance by name.
    raw, err := rpcClient.Dispense("greeter")
    if err != nil {
        log.Fatal(err)
    }

    // Type assertion converts the raw interface to Greeter.
    greeter := raw.(shared.Greeter)

    // Greet invokes the remote method via gRPC.
    msg, err := greeter.Greet(context.Background(), "World")
    if err != nil {
        log.Fatal(err)
    }
    log.Println(msg)
}

The defer client.Kill() call is critical. The plugin runs as a separate process. If you do not kill it, the process leaks and stays alive after the host exits. Always clean up the client.

gofmt will format this code automatically. Trust the tool. Most editors run it on save, so do not argue about indentation.

Walkthrough: Handshake, gRPC, and calls

When the host runs, it executes exec.Command("./plugin"). The plugin binary starts and immediately prints a JSON handshake configuration to standard output. This JSON includes the protocol version, magic cookie, and the gRPC port the plugin is listening on.

The host reads the plugin's standard output, parses the JSON, and verifies the magic cookie. If the keys or values do not match, the host rejects the connection. This prevents loading arbitrary binaries as plugins.

Once the handshake succeeds, the host connects to the gRPC port. The library uses reflection to inspect the Greeter interface and generates the gRPC service definition on the fly. You never see the protobuf code; the library handles serialization and deserialization of arguments and return values.

When you call greeter.Greet, the host sends a gRPC request to the plugin. The plugin receives the request, calls the Greet method on greeterImpl, and sends the result back. To your code, this looks like a local function call. The network hop is invisible.

If the plugin panics, the gRPC connection breaks. The host receives an error like rpc error: code = Unavailable desc = transport is closing. The host process survives. You can catch the error and decide whether to restart the plugin or fail gracefully.

The plugin is a process, not a package. Isolation is the feature.

Realistic example: Lifecycle and signals

Real applications need to handle shutdowns and errors gracefully. The plugin should stop when the host exits, and the host should wait for the plugin to finish if needed. Use signal.NotifyContext to listen for OS signals and propagate them to the plugin.

Here is a lifecycle-aware host that handles graceful shutdown. It sets up the client and signal handling.

// host/lifecycle.go
package main

import (
    "context"
    "log"
    "os"
    "os/signal"
    "syscall"
    "github.com/hashicorp/go-plugin"
    "example.com/myapp/shared"
)

// runPlugin starts the plugin and handles graceful shutdown.
func runPlugin() {
    client := newPluginClient()

    // Client() blocks until the plugin is ready.
    rpcClient, err := client.Client()
    if err != nil {
        log.Printf("failed to connect: %v", err)
        return
    }

    // Kill() terminates the plugin process.
    defer client.Kill()

    // NotifyContext listens for OS signals.
    ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
    defer stop()

    // Goroutine waits for shutdown signal.
    go func() {
        <-ctx.Done()
        log.Println("shutting down...")
    }()

    // Dispense retrieves the plugin instance.
    raw, err := rpcClient.Dispense("greeter")
    if err != nil {
        log.Printf("dispense failed: %v", err)
        return
    }

    // Type assertion converts raw interface to typed Greeter.
    greeter := raw.(shared.Greeter)

    // Use the plugin with the shutdown-aware context.
    msg, err := greeter.Greet(ctx, "World")
    if err != nil {
        log.Printf("greet failed: %v", err)
        return
    }
    log.Println(msg)
}

The defer client.Kill() ensures the plugin process is terminated when the function returns. The signal handler allows you to cancel long-running plugin calls if the user presses Ctrl+C. Pass the context to plugin methods so they can respect cancellation.

context.Context always goes as the first parameter, conventionally named ctx. Functions that take a context should respect cancellation and deadlines.

Always kill the client. A leaked plugin process is a zombie that eats memory.

Pitfalls and errors

Version mismatches are the most common issue. The host and plugin must import the exact same version of the shared package. If the host uses v1.2.0 and the plugin uses v1.3.0, and the interface changed, the reflection layer may fail or data may serialize incorrectly. Lock your shared dependencies and rebuild both binaries together when the interface changes.

The compiler rejects the code with undefined: shared if you forget to import the shared package. At runtime, you get plugin: handshake failed: magic cookie mismatch if the handshake keys differ between host and plugin. You get plugin: plugin not found if you request a plugin name that was not registered.

If the plugin binary does not exist or fails to start, client.Client() returns an error. Check the error before calling Dispense. If the plugin panics, the host receives a gRPC error. The host does not crash, but the plugin process dies. You may need to implement a restart loop if the plugin is critical.

Context deadlines matter. If the plugin hangs, the host blocks until the context expires. Always pass a context with a deadline or cancellation to plugin calls. The worst goroutine bug is the one that never logs.

if err != nil { return err } is verbose by design. The community accepts the boilerplate because it makes the unhappy path visible. Do not hide errors in plugin calls.

Version mismatches are silent killers. Lock your shared dependencies.

Decision: When to use go-plugin

Use hashicorp/go-plugin when you need to load external binaries at runtime and want process isolation so a plugin crash cannot take down the host.

Use Yaegi when you need to execute dynamic Go scripts or expressions without compiling a binary, accepting the performance overhead of interpretation.

Use static interfaces and dependency injection when your extensions are known at compile time and you want zero runtime overhead.

Use cgo and dynamic linking when you are integrating with existing C libraries and need direct memory access, accepting the loss of portability and safety.

Isolation costs CPU. Pick the right tool for the threat model.

Where to go next