tensorflow/model-optimization

A toolkit to optimize ML models for deployment for Keras and TensorFlow, including quantization and pruning.

What it solves

It provides a suite of tools to optimize machine learning models, making them more efficient for deployment and execution.

How it works

The toolkit uses techniques such as quantization and pruning for sparse weights to reduce model size and improve performance. It includes stable Python APIs and specific integrations for Keras.

Who it’s for

It is designed for both novice and advanced users who need to optimize their models for deployment.

Highlights

  • Quantization support
  • Pruning for sparse weights
  • Dedicated APIs for Keras
  • Stable Python APIs

Related

  • Project
    SonySemiconductorSolutions/mct-model-optimization

    MCT is a model compression toolkit from Sony Semiconductor Solutions that quantizes and prunes trained PyTorch and Keras models for efficient edge deployment, offering PTQ, GPTQ, QAT, mixed-precision search, and hardware-aware optimization.

  • Project
    PrunaAI/pruna

    A model optimization framework that uses compression algorithms like quantization and pruning to make AI models faster, smaller, and cheaper to run.

  • Project
    vllm-project/llm-compressor

    LLM Compressor is a Python library that quantizes and compresses large language models for fast, low‑memory inference with vLLM. It supports many low‑bit formats (FP8, FP4, INT4), a variety of algorithms (GPTQ, AWQ, AutoRound, REAP), and can handle multi‑trillion‑parameter models via disk off‑loading and DDP. The library integrates directly with Hugging Face checkpoints, saves models in the `compressed‑tensors` format, and provides ready‑to‑use examples and pre‑compressed checkpoints.

  • Project
    fairlearn/fairlearn

    Fairlearn is a Python library that provides tools for data scientists to assess and mitigate bias in AI models, focusing on group fairness and the reduction of allocation or quality-of-service harms.

  • Project
    rasbt/mlxtend

    A Python library providing machine learning extensions for data science tasks, including ensemble methods, feature selection, and visualization utilities.