huggingface/ratchet

A cross-platform browser ML framework.

What it solves

Ratchet provides a toolkit for developers to integrate fast, GPU-accelerated ML inference directly into native and browser-based applications without needing a backend server.

How it works

It utilizes WebGPU and CPU for computation, focusing on inference-only workloads. The toolkit supports first-class quantization and lazy computation to ensure high performance and efficiency, with a focus on being "inplace by default."

Who it’s for

ML developers who want to deploy performant AI functionality into production applications across different platforms.

Highlights

  • GPU acceleration via WebGPU
  • Cross-platform support for native and browser environments
  • Support for models like Whisper, Phi 2 & 3, and Moondream
  • Built-in quantization support
  • Asynchronous loading and caching using IndexedDB

Related

  • Project
    bayesflow-org/bayesflow

    A Python library for efficient amortized Bayesian inference using deep learning, supporting multiple backends like JAX, PyTorch, and TensorFlow.

  • Project
    NucleoidAI/Nucleoid

    A declarative logic programming language and runtime for building world models in Neuro-Symbolic AI, designed to reduce LLM hallucinations through structured reasoning.

  • Project
    SharpAI/SwiftLM

    A native Swift inference server for Apple Silicon that serves MLX models via an OpenAI-compatible API, featuring SSD expert streaming and KV cache compression.

  • Project
    claw-eval/claw-eval

    A trustworthy evaluation framework for autonomous agents featuring 300 human-verified tasks and a strict Pass^3 metric to ensure consistent model performance.