onnx/onnx-mlir

Representation and Reference Lowering of ONNX Models in MLIR Compiler Infrastructure

What it solves

ONNX-MLIR provides a way to transform Open Neural Network Exchange (ONNX) graphs into executable code with minimal runtime support. This allows AI models to be deployed as highly optimized, standalone libraries or binaries rather than relying on a heavy runtime environment.

How it works

Built on the LLVM/MLIR compiler technology, the project implements an ONNX Dialect. It uses a driver (onnx-mlir) to lower ONNX graphs through various stages of representation, allowing users to emit different outputs such as MLIR files, LLVM bytecodes, or shared libraries in C, C++, and Java.

Who it’s for

It is designed for developers and engineers who need to compile ONNX models into optimized machine code for specific hardware, such as generic CPUs or IBM's Telum integrated AI accelerator.

Highlights

  • Multi-language support: Generates libraries and runtimes for Python, C, C++, and Java.
  • Flexible output formats: Can emit ONNX IR, MLIR, LLVM IR, object files, or shared libraries.
  • Optimization levels: Includes built-in optimization levels (O0 through O3) to tune performance.
  • Hardware targets: Specifically supports generic CPUs and IBM Telum NNPA.

Related

  • Project
    kraiskil/onnx2c

    A compiler that converts ONNX models into standalone C code for efficient neural network inference on microcontrollers.

  • Project
    sophgo/tpu-mlir

    An MLIR-based machine-learning compiler that converts pre-trained neural networks from frameworks like PyTorch and ONNX into optimized bmodel files for TPU execution.

  • Project
    onnx/optimizer

    A C++ library for performing arbitrary optimizations on ONNX models, providing shared optimization passes to be reused across different backend implementations.

  • Project
    inisis/OnnxSlim

    A tool for simplifying and slimming ONNX models to reduce operator count and improve inference speed while maintaining accuracy.

  • Project
    microsoft/onnxscript

    A Python-based framework that allows developers to author, debug, and optimize ONNX models and functions using a concise subset of Python.