apache/tvm-ffi

Open ABI and FFI for Machine Learning Systems

What it solves

It addresses the fragmentation in machine learning systems where kernel libraries, DSLs, and runtimes often require separate implementations or bindings for different frameworks (like PyTorch or JAX) and languages (like Python, C++, or Rust). It provides a unified, stable way to share machine learning kernels and code across these different environments without needing to write custom glue code for every combination.

How it works

TVM FFI establishes a minimal, framework-agnostic C ABI (Application Binary Interface) and Foreign Function Interface. It uses a compact value and call convention to minimize overhead and implements zero-copy data interchange across frameworks using the DLPack protocol.

Who it’s for

  • Kernel developers who want to distribute a single package (wheel) that works across multiple ML frameworks and Python versions.
  • DSL creators building JIT or AOT kernel exposure frameworks.
  • Framework and runtime developers seeking a uniform extension point for ABI-compliant libraries.
  • ML infrastructure engineers needing consistent interop across Python, C++, and Rust.

Highlights

  • Stable C ABI: Prioritizes a consistent binary interface for long-term compatibility.
  • Zero-copy interop: Enables efficient data sharing between PyTorch, JAX, and CuPy via DLPack.
  • Multi-language support: Out-of-the-box bindings for Python, C++, and Rust.
  • Framework agnostic: Works across major ML ecosystems including PyTorch, JAX, PaddlePaddle, and NumPy/CuPy.

Related

  • Project
    tashfeenahmed/freellmapi

    A local router that aggregates free tiers from dozens of LLM providers into a single OpenAI-compatible endpoint, providing billions of tokens of free inference capacity.

  • Project
    Mesh-LLM/mesh-llm

    A distributed LLM serving system that pools GPUs and memory across multiple machines to run large models via an OpenAI-compatible API.

  • Project
    google-parfait/tensorflow-federated

    An open-source framework for machine learning and computations on decentralized data, enabling global model training while keeping training data local to clients.

  • Project
    HorizonRobotics/alf

    ALF (Agent Learning Framework) is a PyTorch‑based reinforcement‑learning library offering a large collection of algorithms (A2C, PPO, SAC, MuZero, etc.), modular components, Python‑style configuration, and support for single‑ and multi‑GPU/cluster training. It includes many ready‑to‑run examples (CartPole, Atari, MuJoCo, MetaDrive) and provides Docker, Nix, and uv‑based installation options.

  • Project
    ZHZisZZ/dllm

    A library that unifies the training, inference, and evaluation of diffusion language models, providing scalable pipelines and recipes for converting AR models into diffusion models.