yalue/onnxruntime_go

What it solves

This library provides a simple Go (Golang) interface for loading and executing neural networks in the ONNX format. It specifically addresses the challenge of cross-platform compatibility, enabling ONNX Runtime's C/C++ APIs to be used in Go on Windows, Linux, and macOS, overcoming common compiler conflicts between CGo and MSVC on Windows.

How it works

Instead of relying on the onnxruntime source code, the library manually loads the onnxruntime shared library (DLL or .so) at runtime. This removes the dependency on specific compilers and allows the library to use Go generics to support various Tensor data types. It supports multiple execution providers for hardware acceleration, including CUDA, TensorRT, and CoreML.

Who it’s for

Go developers who want to integrate pre-trained ONNX neural networks into their applications without needing to write C++ code or having to deal with complex cross-platform build configurations.

Highlights

• Cross-Platform Support: Works across Windows, Linux, and macOS.
• Hardware Acceleration: Confirmed support for CUDA, TensorRT, and CoreML execution providers.
• Generic Tensors: Uses Go generics to handle multiple Tensor data types efficiently.
• Flexible Session Management: Offers AdvancedSession for precise control over input/output tensors and DynamicAdvancedSession for flexible tensor specification during execution.