pykeio/ort

Fast ML inference & training for ONNX models in Rust

What it solves

It provides a way for Rust developers to run machine learning models in the ONNX format with hardware acceleration. This allows models trained in frameworks like PyTorch, TensorFlow, Keras, scikit-learn, or PaddlePaddle to be deployed efficiently on a variety of devices, from data centers to end-user hardware.

How it works

ort acts as a Rust interface (wrapper) for Microsoft's ONNX Runtime library, while also supporting other pure-Rust runtimes. It enables the execution of ONNX models across almost any hardware accelerator available.

Who it’s for

Rust developers who need to deploy pre-trained machine learning models on-device or in the cloud with high performance and hardware acceleration.

Highlights

  • Hardware-accelerated inference and training.
  • Supports a wide range of hardware accelerators via ONNX Runtime.
  • Compatible with models from PyTorch, TensorFlow, Keras, scikit-learn, and PaddlePaddle.
  • Lightweight enough for on-device deployment.

Related

  • Project
    robertknight/rten

    A machine learning runtime written in Rust that enables running ONNX models trained in PyTorch and other Python frameworks in Rust and WebAssembly environments.

  • Project
    jamjamjon/usls

    A cross-platform Rust library powered by ONNX Runtime for efficient inference of vision and vision-language models under 1B parameters.

  • Project
    microsoft/onnxruntime

    A cross-platform machine learning accelerator that optimizes inference and training for models from various frameworks like PyTorch and TensorFlow.

  • Project
    altunenes/parakeet-rs

    parakeet‑rs is a Rust crate that wraps NVIDIA’s Parakeet/Nemotron ONNX speech‑recognition and diarisation models, offering offline and streaming transcription (English‑only, multilingual, multi‑speaker) with optional GPU acceleration and an experimental pure‑Rust backend.

  • Project
    onnx/onnx-tensorrt

    A TensorRT backend for ONNX that parses ONNX models for high-performance execution on NVIDIA GPUs.