google/ml-compiler-opt

Infrastructure for Machine Learning Guided Optimization (MLGO) in LLVM.

What it solves

It replaces human-crafted optimization heuristics in the LLVM compiler with machine learning models to improve how the compiler optimizes code for size and performance.

How it works

MLGO integrates ML techniques into LLVM to replace traditional heuristics. It uses algorithms like Policy Gradient to train policies for specific compiler tasks. The framework currently focuses on two main optimizations: inlining-for-size and register-allocation-for-performance.

Who it’s for

Compiler engineers and developers who want to optimize LLVM's compilation process using machine learning models rather than manual heuristics.

Highlights

  • Supports inlining-for-size and register-allocation-for-performance optimizations.
  • Uses Policy Gradient training for policies.
  • Provides training infrastructure and tools to create custom models.
  • Offers pretrained models that can be integrated directly into LLVM via CMake flags.

Related

  • Project
    sophgo/tpu-mlir

    An MLIR-based machine-learning compiler that converts pre-trained neural networks from frameworks like PyTorch and ONNX into optimized bmodel files for TPU execution.

  • Project
    allenai/OLMo-core

    A library of building blocks and training infrastructure for the OLMo family of large language models, providing official training scripts and optimized hardware kernels.

  • Project
    evidentlyai/evidently

    An open-source Python framework to evaluate, test, and monitor ML and LLM-powered systems, helping developers detect data drift and ensure output quality.

  • Project
    context-labs/HALO

    An RLM-based agent optimizer that analyzes production execution traces to identify systemic failure modes and provide recommendations for improving agent harnesses.

  • Project
    vllm-project/llm-compressor

    LLM Compressor is a Python library that quantizes and prunes large language models into the `compressed‑tensors` format, enabling memory‑efficient deployment with vLLM. It supports many low‑precision formats (NVFP4, FP8, INT4, etc.), several PTQ/GPTQ algorithms, DDP and disk‑offloading for huge models, and ships pre‑quantized checkpoints for popular LLMs.