buddy-compiler/buddy-mlir

What it solves

Buddy MLIR is a compiler framework designed to bridge the gap between high-level domain-specific languages (DSLs) and domain-specific architectures (DSAs), enabling the efficient compilation and deployment of AI models on specialized hardware.

How it works

Built on top of MLIR (Multi-Level Intermediate Representation), the framework provides a pipeline for converting AI models into optimized machine code. It includes a codegen tool (build_model.py) that takes model specifications (JSON) and weights (e.g., from HuggingFace) to generate executable model files (.rax). These files can then be run via a dedicated CLI tool (buddy-cli) for inference, supporting advanced features like tiered KV caches and layer-partitioned compilation.

Who it’s for

It is intended for compiler engineers, hardware architects, and AI researchers working on co-designing software and hardware for deep learning acceleration.

Highlights

• Broad Model Support: Provides specifications and build paths for major models including DeepSeek R1, Whisper, and Qwen3-VL.
• Hardware Flexibility: Supports multiple targets including x86_64, RISC-V, and NVIDIA GPUs (NVPTX).
• Advanced Compilation: Supports template-based layer-partitioned compilation to optimize how models are split across hardware.
• End-to-End Workflow: Includes tools for importing weights from local HuggingFace directories and running inference directly via a CLI.