How to Use TensorFlow or ONNX Runtime from Go

Ai Sovereignty
2026-04-17T16:32:01+00:00
You cannot use TensorFlow directly from Go because it lacks an official Go API, but you can easily use ONNX Runtime via its official Go bindings to execute pre-trained models.

You cannot use TensorFlow directly from Go because it lacks an official Go API, but you can easily use ONNX Runtime via its official Go bindings to execute pre-trained models. For TensorFlow models, you must first convert them to the ONNX format and then load them using the ONNX Runtime Go library.

The standard workflow involves exporting your TensorFlow model to ONNX using Python, then loading and running it in your Go application. This approach leverages Go's strong concurrency model while relying on the high-performance C++ backend of ONNX Runtime.

First, convert your TensorFlow model to ONNX using the tf2onnx Python library. This step is required because Go cannot parse TensorFlow's .pb or SavedModel formats natively.

# Install the converter
pip install tf2onnx

# Convert a TensorFlow SavedModel to ONNX
python -m tf2onnx.convert \
  --saved-model ./my_model \
  --output my_model.onnx \
  --opset 17

Next, implement the inference logic in Go using the onnxruntime package. You will need to initialize the environment, load the session, and manage input/output tensors manually.

package main

import (
	"fmt"
	"log"

	"github.com/yalue/onnxruntime-go"
)

func main() {
	// Initialize the ONNX Runtime environment
	env, err := onnxruntime.NewEnvironment("CPU", "TensorFlow-Model")
	if err != nil {
		log.Fatal(err)
	}
	defer env.Release()

	// Create a session with the ONNX model
	session, err := env.NewSession("my_model.onnx", nil)
	if err != nil {
		log.Fatal(err)
	}
	defer session.Release()

	// Prepare input data (example: float32 array)
	inputData := []float32{1.0, 2.0, 3.0, 4.0}
	inputTensor, err := onnxruntime.NewTensorFromFloat32(inputData, []int64{1, 4})
	if err != nil {
		log.Fatal(err)
	}
	defer inputTensor.Release()

	// Run inference
	outputs, err := session.Run([]string{"input_tensor"}, []onnxruntime.Value{inputTensor}, []string{"output_tensor"})
	if err != nil {
		log.Fatal(err)
	}
	defer outputs[0].Release()

	// Extract results
	result := outputs[0].AsFloat32()
	fmt.Printf("Inference result: %v\n", result)
}

Note that you must install the onnxruntime C++ library on your system (via apt, brew, or vcpkg) and ensure the Go package can locate the shared library. The Go bindings are wrappers around the native C++ API, so performance is comparable to Python or C++ implementations. While TensorFlow Lite offers a Go plugin, it is less flexible for complex models than the ONNX Runtime approach.