llvm/torch-mlir

The Torch-MLIR project aims to provide first class support from the PyTorch ecosystem to the MLIR ecosystem.

What it solves

It solves the problem of software fragmentation and high development costs for hardware vendors who want to support PyTorch. Instead of every vendor building their own custom frontend to translate PyTorch models into a format their hardware understands, this project provides a standardized compiler bridge to the MLIR ecosystem.

How it works

Torch-MLIR acts as a middle layer that lowers PyTorch models into the Torch-MLIR Dialect. It supports two primary entry points for this process: ONNX and Fx. Once in the MLIR ecosystem, these models can be more easily mapped to specific target hardware by downstream compilers.

Who it’s for

It is primarily designed for compiler developers and hardware vendors who need to integrate PyTorch support into their systems without implementing a full PyTorch frontend from scratch.

Highlights

  • Provides first-class compiler support from PyTorch to MLIR.
  • Supports multiple entry paths including ONNX and Fx.
  • Reduces the need for redundant custom lowerings by hardware vendors.
  • Integrates with downstream projects like IREE and Blade.

Related

  • Project
    sophgo/tpu-mlir

    An MLIR-based machine-learning compiler that converts pre-trained neural networks from frameworks like PyTorch and ONNX into optimized bmodel files for TPU execution.

  • Project
    onnx/onnx-mlir

    A compiler that transforms ONNX graphs into optimized, low-runtime code using LLVM/MLIR technology for efficient model deployment.

  • Project
    google/heir

    An MLIR-based compiler toolchain for homomorphic encryption that allows developers to write encrypted computations and deploy them across multiple HE libraries and schemes.

  • Project
    Lightning-AI/lightning-thunder

    A source-to-source compiler for PyTorch that optimizes models for training and inference through a Pythonic intermediate representation and extensible transformation plugins.

  • Project
    iree-org/iree

    IREE is an MLIR-based compiler and runtime that lowers machine learning models to a unified IR for efficient deployment across data centers, mobile, and edge devices.