yalue/onnxruntime_go

What it solves

Go developers who want to run ONNX neural networks from their applications face a challenge: the official ONNX Runtime library primarily offers C and C++ APIs, and existing Go wrappers often fail on Windows because they rely on the MSVC compiler while Go's cgo on Windows requires Mingw. This library removes that friction by loading the ONNX Runtime shared library at runtime, so the same Go code can load and run ONNX models on Windows, Linux, macOS, and ARM64 systems without compiling against the ONNX Runtime source code.

How it works

The wrapper uses cgo only for the Go glue, but it does not link against ONNX Runtime directly. Instead, it calls SetSharedLibraryPath to point to the correct onnxruntime.so, .dll, or .dylib for the current platform, then manually loads the functions from that shared library using the ONNX C API headers. This approach makes Windows work and keeps the same pattern on all platforms. The library also leverages Go's generics to define typed tensors (NewTensor, NewEmptyTensor) so you can work with common data types like float32 without extra conversion. Sessions are created with NewAdvancedSession, where you supply input and output tensor names and pre‑allocated tensors; calling Run() executes the network and updates the output tensors. For rapid iteration, there is also a DynamicAdvancedSession that lets you supply tensors at Run() time instead of at session creation.

Who it’s for

Go developers who want to integrate ONNX neural network inference into their applications, especially those targeting multiple operating systems including Windows. It’s also for teams that need to work with specific ONNX Runtime versions and want a straightforward, version‑pinned setup. The library is simple enough for a small script but supports advanced execution providers like CUDA, TensorRT, and CoreML when the underlying ONNX Runtime build includes them.

Highlights

• Cross‑platform support: Works on Windows, Linux, macOS, and ARM64 by loading the shared library at runtime instead of linking.
• Generics‑based tensors: NewTensor and NewEmptyTensor use Go generics, so you can use native Go slices for common data types without boilerplate.
• Pre‑allocated tensors for performance: Create input and output tensors once, then reuse them across multiple Run() calls for a slight speed boost.
• Execution provider support: Confirmed compatibility with CUDA, TensorRT, and CoreML, with tests that automatically skip accelerator tests when the library doesn’t include them.
• Version control: The project pins ONNX Runtime 1.31.0 and provides a straightforward upgrade path by replacing header files and the shared library.