onnxsim/onnxsim

Simplify your onnx model

What it solves

ONNX Simplifier reduces the complexity of ONNX models by removing redundant operators and simplifying the computation graph. This is particularly useful when exporting models from frameworks like PyTorch, which often introduce unnecessary shape-manipulation nodes that make the model harder to read, optimize, and deploy.

How it works

The tool runs a fixed-point loop of shape inference, graph optimization, and constant folding. It evaluates constant parts of the graph and replaces them with their computed outputs. It also propagates tensor shapes through the graph to unlock further folding and can optionally validate the simplified model against the original using random inputs to ensure correctness.

Who it’s for

It is designed for AI engineers and ML developers who need to optimize ONNX models for deployment on various hardware targets, including those using MLIR-based compilers, Apple Core ML, or TensorFlow Lite.

Highlights

  • Constant Folding: Replaces redundant operators with their computed constant outputs.
  • Graph Optimization: Integrates with onnx-optimizer for fusions (e.g., BatchNorm into Conv) and eliminations.
  • Broad Export Support: Direct export paths to MLIR (torch-mlir/onnx-mlir), Core ML, and TensorFlow Lite.
  • Framework Integration: Specialized export functions for Hugging Face Transformers, Diffusers, Detectron2, and SAM 2.
  • Multi-platform: Available as a Python package, CLI, C API, Rust wrapper, and a zero-install web version.
  • Quantization Support: Includes quantization-aware fine-tuning (QAT) to recover accuracy lost during quantization.

Related

  • Project
    inisis/OnnxSlim

    A tool for slimming down ONNX models by reducing operators to improve inference speed and load times without sacrificing accuracy.

  • Project
    onnx/onnx

    ONNX is an open, framework‑agnostic model format and reference implementation that lets you move AI models between tools and runtimes. Install via `pip install onnx`, use the Python API to build or load a graph, then run inference with any ONNX‑compatible engine.

  • Project
    pnnx/pnnx

    pnnx is an open‑source tool that optimises PyTorch models and exports them to a lightweight, dependency‑free format (PNNX) and to ncnn/ONNX‑zero files, enabling fast inference on edge devices.

  • Project
    onnx/onnx-mlir

    A compiler that transforms ONNX graphs into optimized, low-runtime code using LLVM/MLIR technology for efficient model deployment.

  • Project
    microsoft/Olive

    An AI model optimization toolkit that composes the best optimization techniques to create efficient ONNX models for cloud or edge inference based on hardware and performance constraints.