buddy-compiler/buddy-mlir

What it is

Buddy MLIR is an open‑source compiler framework built on top of LLVM’s MLIR infrastructure. It lets developers define domain‑specific languages (DSLs) and generate code that runs efficiently on domain‑specific hardware (DSAs) such as CPUs, RISC‑V cores, or NVIDIA GPUs. The project ships a collection of ready‑made model pipelines (e.g., DeepSeek R1, Whisper, Qwen3‑VL) that are compiled down to a custom binary format (.rax) and executed with a tiny runtime (buddy‑cli).

Why it matters

Modern AI models are huge and hardware‑aware; getting the best performance often requires a tight co‑design of the model representation and the target accelerator. Buddy MLIR provides a single place to write high‑level MLIR passes, plug in hardware‑specific lowering, and produce runnable binaries without hand‑crafting CUDA or assembly. This lowers the barrier for researchers and chip designers to experiment with new AI kernels and to benchmark them on emerging architectures like RISC‑V or custom ASICs.

Who should look at it

• AI compiler researchers who need a modular MLIR‑based stack to prototype new dialects or optimizations.
• Hardware architects building domain‑specific accelerators and wanting a way to compile existing large‑language‑model workloads onto them.
• ML engineers interested in running state‑of‑the‑art models (DeepSeek, Whisper, Qwen3‑VL) on non‑standard platforms (RISC‑V, multi‑CPU NUMA nodes, GPUs) with minimal hand‑tuning.
• Students learning about compiler pipelines for deep‑learning workloads; the repo includes end‑to‑end examples and a Python‑friendly API.

How to get started

1. Install system deps – LLVM/MLIR, flatbuffers, libnuma, etc.
2. Clone with submodules and build the bundled LLVM/MLIR using CMake/Ninja.
3. Create a Python venv and install the requirements.txt (pip, uv or conda work).
4. Build Buddy MLIR – point CMake at the just‑built LLVM/MLIR directories, enable Python bindings, and run ninja.
5. Compile a model – run tools/buddy-codegen/build_model.py with a JSON spec (e.g., models/whisper/specs/base.json). Optionally enable layer‑partitioning for template‑based codegen.
6. Run inference – use the generated .rax file with build/bin/buddy-cli, supplying a prompt, audio, or image as required.
7. Optional – build a wheel (scripts/release.sh) for easy pip installation, or enable the end‑to‑end test suite (-DBUDDY_ENABLE_E2E_TESTS=ON).

What you’ll see in the repo

• llvm/ submodule – a trimmed LLVM/MLIR source tree used for building the compiler.
• tools/buddy-codegen/ – Python scripts that translate model specs (JSON) into MLIR, then lower to the custom runtime.
• models/ – ready‑made specs and test data for DeepSeek R1, Whisper, and Qwen3‑VL.
• examples/ – small programs showing IR‑level transformations and how to invoke the runtime.
• scripts/ – helper scripts for releasing wheels and Docker‑based many‑linux builds.
• Documentation – extensive docs/ covering environment setup, layer partitioning, many‑linux notes, and contribution guidelines.

TL;DR

Buddy MLIR is a real, research‑grade compiler stack that bridges high‑level AI models and custom hardware. It gives you a way to compile, optimise, and run large language‑vision models on CPUs, RISC‑V, or GPUs with a unified MLIR‑based workflow.