buddy-compiler/buddy-mlir

What it solves

Buddy MLIR is a compiler framework designed to bridge the gap between high-level domain-specific languages (DSLs) and domain-specific architectures (DSAs). It enables the efficient compilation of AI models—such as DeepSeek R1, Whisper, and Qwen3-VL—into optimized kernels for various hardware targets, including x86_64, RISC-V, and NVIDIA GPUs.

How it works

Built on top of the MLIR (Multi-Level Intermediate Representation) ecosystem, the framework provides a set of IR-level transformations and passes. It uses a codegen tool (build_model.py) to convert model specifications (JSON specs) and weights from sources like HuggingFace into a deployable format (.rax files). It also includes a buddy-cli for running inference on these compiled models, supporting advanced hardware configurations like NUMA node binding and CPU pinning.

Who it’s for

Hardware architects, compiler engineers, and AI researchers who are developing custom AI accelerators or optimizing the deployment of large models on specific hardware architectures.

Highlights

• Multi-Hardware Support: Targets x86_64, RISC-V, and NVIDIA GPUs (NVPTX).
• End-to-End Compilation: Converts model specs and weights directly into executable binaries for inference.
• Layer Partitioning: Parallelizes the slowest MLIR compilation stages to speed up the build process while maintaining correctness.
• Broad Model Support: Includes pre-defined specifications for LLMs (DeepSeek R1), audio models (Whisper), and vision-language models (Qwen3-VL).