qualcomm/aimet

AIMET is a library that provides advanced quantization and compression techniques for trained neural network models.

AI Model Efficiency Toolkit (AIMET)

What it is – AIMET is an open‑source software toolkit from Qualcomm that helps you shrink and speed up deep‑learning models so they can run efficiently on edge devices (phones, laptops, DSPs). It works with PyTorch and ONNX models and provides both post‑training quantization (PTQ) and quantization‑aware training (QAT) pipelines, plus a few model‑compression utilities.

Why it matters – Running inference with integer (INT8/INT4) arithmetic is dramatically faster and uses far less memory than floating‑point (FP32). AIMET automates the often‑tricky process of converting a trained model to low‑precision while keeping accuracy loss minimal. The toolkit includes advanced techniques such as Data‑Free Quantization, AdaRound, Cross‑Layer Equalization, and spatial SVD pruning, which let you get 5‑15× speedups on Qualcomm Hexagon DSPs and up to 4× smaller model footprints.

Key capabilities

  • PTQ techniques – Calibration, AdaRound, SeqMSE, BatchNorm folding/re‑estimation, Cross‑Layer Equalization, AdaScale, OmniQuant, SpinQuant, etc. (supported for both ONNX and PyTorch where indicated).
  • QAT support – Integrated with aimet‑torch so you can continue training a model while simulating quantization effects.
  • Model compression – Spatial SVD decomposition, channel pruning, and automatic per‑layer compression‑ratio selection.
  • Visualization tools – Inspect weight ranges and layer‑wise compression sensitivity to guide optimization.
  • Edge‑ready output – Quantized models can be deployed on Qualcomm Hexagon DSP, Kryo CPU, or any platform that runs integer inference.

How to get started – Pre‑built wheels are published on PyPI (aimet-onnx, aimet-torch). The README points to a quick‑start guide and Docker‑based build instructions for those who want the latest source.

Results snapshot – The toolkit can quantize MobileNet‑v2, ResNet‑50, DeepLab‑v3, ADAS object‑detect, and even recurrent models like DeepSpeech2 with less than 1 % accuracy loss (e.g., MobileNet‑v2 FP32 top‑1 71.72 % → INT8 71.08 %). AdaRound can recover accuracy for hard‑to‑quantize models and even enable 4‑bit weight quantization with modest loss.

Community & licensing – Discussions happen on GitHub and a dedicated Slack channel. Contributions are welcomed under a BSD‑3‑Clause license.

Useful links