jamjamjon/usls

A Rust library integrated with ONNXRuntime, providing a collection of Computer Vison and Vision-Language models such as YOLO, FastVLM, and more.

What it solves

usls is a cross-platform Rust library designed for the efficient inference of state-of-the-art (SOTA) vision and vision-language models, specifically targeting those typically under 1 billion parameters.

How it works

It leverages the ONNX Runtime to provide a unified API for model inference. The library includes a full-stack suite of tools including a DataLoader, Annotator, and Viewer to handle complete workflows. It supports a wide range of execution providers (such as CUDA, TensorRT, CoreML, and OpenVINO) and multiple precision formats (FP32, FP16, INT8, Q4, etc.) to optimize performance across different hardware.

Who it’s for

Developers who want to integrate high-performance vision and vision-language models into Rust applications across Linux, macOS, and Windows.

Highlights

  • Extensive Model Zoo: Supports over 50 SOTA models across tasks like object detection (YOLO series, RT-DETR), image segmentation (SAM, FastSAM), pose estimation, and OCR.
  • High Performance: Utilizes multi-threading, SIMD, and CUDA acceleration.
  • Cross-Platform Support: Compatible with major OSs and hardware accelerators via ONNX Runtime.
  • Automated Management: Handles model downloading from HuggingFace/GitHub, caching, and path resolution automatically.
  • Flexible Inputs: Supports images, directories, videos, webcams, and streams.

Related

  • Project
    pykeio/ort

    A Rust interface for hardware-accelerated inference and training of ONNX machine learning models, enabling efficient deployment across various hardware and platforms.

  • Project
    robertknight/rten

    A machine learning runtime written in Rust that enables the execution of ONNX models trained in PyTorch on CPUs and WebAssembly.

  • Project
    amd/RyzenAI-SW

    A suite of tools and runtime libraries for optimizing and deploying AI inference on AMD Ryzen AI-based PCs to maximize hardware acceleration.

  • Project
    Anush008/fastembed-rs

    A Rust library for local vector embedding generation and reranking, supporting text, images, and sparse embeddings via ONNX and Candle backends.

  • Project
    lablup/mlxcel

    A high-performance Rust-based inference runtime and server for LLMs and VLMs on Apple Silicon and NVIDIA CUDA, removing the need for Python environments during deployment.