knights-analytics/hugot

What it solves

Hugot provides a way to run transformer-based AI models natively in Go applications. It eliminates the need to maintain separate Python RPC services or rely on external REST APIs for inference, allowing developers to tightly couple ML models with their Go production stacks for better performance and scalability.

How it works

The library uses ONNX (Open Neural Network Exchange) as its primary model format. It allows models trained in Python via the Hugging Face transformers library to be exported to ONNX and then run in Go. Hugot supports three pluggable backends for execution:

• Go: A native Go backend for simpler workloads and environments without cgo.
• ONNX Runtime (ORT): A high-performance C backend that supports all pipelines, including generative AI.
• OpenXLA: A C backend that supports TPUs and training/fine-tuning capabilities.

Who it’s for

It is designed for Go developers and ML engineers who want to deploy and fine-tune transformer pipelines on their own hardware without writing custom inference or training code from scratch.

Highlights

• Hugging Face Compatibility: Lift-and-shift models from Python to Go with identical predictions.
• Diverse Pipeline Support: Includes text generation, image classification, object detection, question answering, and zero-shot classification.
• Hardware Acceleration: Supports CUDA, TPU, TensorRT, DirectML, CoreML, and OpenVINO.
• Training Capabilities: Supports fine-tuning of feature extraction pipelines (e.g., for RAG and semantic search) using the XLA backend.
• Pluggable Storage: Allows replacing the default OS filesystem with custom adapters for object stores.