inisis/OnnxSlim

A Toolkit to Help Optimize Onnx Model

What it solves

OnnxSlim reduces the complexity of ONNX models by removing unnecessary operators. This process slims down the model while maintaining the same accuracy, which leads to improved inference speed and faster model loading times.

How it works

It provides a Python package and a command-line interface to process ONNX models. Users can either run a simple bash command to slim a model or use the onnxslim.slim() function within a Python script to optimize the graph structure of the model.

Who it’s for

It is designed for AI developers and ML engineers who need to optimize their models for deployment, particularly those using the ONNX format to target various hardware accelerators or inference engines.

Highlights

  • Integrated into major frameworks like NVIDIA TensorRT-Model-Optimizer, HuggingFace Optimum, and transformers.js.
  • Proven performance in LLM inference optimization challenges (ranking 1st in AICAS 2024 and 2025).
  • Widely adopted by projects such as Ultralytics and MNN-LLM.
  • Simple installation via pip.

Related

  • Project
    onnxsim/onnxsim

    A tool to simplify ONNX models by performing constant folding and graph optimization to remove redundant operators and reduce complexity.

  • Project
    microsoft/onnxruntime

    A cross-platform machine learning accelerator that optimizes inference and training for models from various frameworks like PyTorch and TensorFlow.

  • Project
    microsoft/Olive

    An AI model optimization toolkit that composes the best optimization techniques to create efficient ONNX models for cloud or edge inference based on hardware and performance constraints.

  • Project
    NVIDIA/Model-Optimizer

    NVIDIA Model Optimizer is an open‑source Python library for compressing and speeding up large AI models (LLMs, VLMs, diffusion). It supports PTQ, quant‑aware training, pruning, distillation, speculative decoding, and sparsity, works with Hugging Face, PyTorch, or ONNX inputs, and exports directly to TensorRT‑LLM, vLLM, SGLang, etc. Install via `pip install nvidia-modelopt[all]` or use NVIDIA containers. The repo includes docs, examples, and pre‑quantized checkpoints.

  • Project
    microsoft/onnxscript

    A Python-based framework that allows developers to author, debug, and optimize ONNX models using a natural Python syntax instead of manual graph construction.