inisis/OnnxSlim

A Toolkit to Help Optimize Onnx Model

What it solves

OnnxSlim reduces the complexity of ONNX models by removing unnecessary operators. This process "slims" the model to improve inference speed and reduce the number of operators without sacrificing the model's accuracy.

How it works

It provides a Python package and a command-line interface (CLI) that allows users to process an existing ONNX model. By applying optimization passes, it simplifies the graph structure of the model, resulting in a more efficient version of the same model.

Who it’s for

AI developers and ML engineers who use ONNX models for deployment, specifically those looking to optimize performance and load speeds on various hardware targets.

Highlights

  • Widespread Integration: Integrated into major frameworks like NVIDIA TensorRT-Model-Optimizer, HuggingFace optimum, and transformers.js.
  • Proven Performance: Ranked 1st in the AICAS LLM inference optimization challenges for both 2024 and 2025.
  • Easy to Use: Offers both a simple one-line bash command and a Python API for integration into scripts.
  • High Adoption: Has achieved over 10 million downloads on PyPI.

Related

  • Project
    onnxsim/onnxsim

    A tool to simplify ONNX models by performing constant folding and graph optimization to remove redundant operators and reduce complexity.

  • Project
    microsoft/onnxruntime

    A cross-platform machine learning accelerator that optimizes inference and training for models from various frameworks like PyTorch and TensorFlow.

  • Project
    microsoft/Olive

    An AI model optimization toolkit that composes the best optimization techniques to create efficient ONNX models for cloud or edge inference based on hardware and performance constraints.

  • Project
    microsoft/onnxscript

    A Python-based framework that allows developers to author, debug, and optimize ONNX models and functions using a concise subset of Python.

  • Project
    NVIDIA/Model-Optimizer

    NVIDIA Model Optimizer (ModelOpt) is an open‑source Python library that lets you apply quantization, pruning, NAS, distillation, speculative decoding, and sparsity to Hugging Face, PyTorch, or ONNX models and export them for fast inference with NVIDIA runtimes like TensorRT‑LLM, vLLM, and SGLang.