zml/zml

Any model. Any hardware. Zero compromise. Built with @ziglang / @openxla / MLIR / @bazelbuild

What it solves

ZML 是一個生產環境推論堆疊，旨在將 AI 工作負載與專有硬體解耦。它讓開發者能在各種硬體加速器上執行模型，而不必為每個平台重寫程式碼庫。

How it works

使用 Zig 語言、MLIR 與 Bazel 建置，ZML 直接將模型編譯至多種硬體後端。它支援包括 NVIDIA CUDA、AMD ROCm、Intel OneAPI、Google TPU 與 AWS Trainium/Inferentia 2 在內的廣泛加速器，確保在任意選定硬體上達到最佳效能。

Who it’s for

此工具適合構建生產 AI 推論系統的開發者與工程師，讓他們避免硬體鎖定，並維持單一程式碼庫以支援多平台部署。

Highlights

  • Multi-Hardware Support: Native compilation for NVIDIA, AMD, Intel, and TPU/Trainium accelerators.
  • Unified Codebase: Run any model on many hardwares using one codebase.
  • LLM Support: Out-of-the-box support for Llama 3.1/3.2, Qwen 3.5, and LFM 2.5.
  • Flexible Loading: Ability to load models from Hugging Face, S3, or local directories via a VFS layer.
  • High Performance: Compiled directly to hardware for peak execution speed.