zml/zml

Any model. Any hardware. Zero compromise. Built with @ziglang / @openxla / MLIR / @bazelbuild

What it solves

ZML 是一个生产推理栈，旨在将 AI 工作负载与专有硬件解耦。它允许开发者在各种硬件加速器上运行模型，而无需为每个平台重写代码库。

How it works

使用 Zig 语言、MLIR 和 Bazel 构建，ZML 直接将模型编译到多个硬件后端。它支持包括 NVIDIA CUDA、AMD ROCm、Intel OneAPI、Google TPU 和 AWS Trainium/Inferentia 2 在内的广泛加速器，确保在任意选定硬件上实现最佳性能。

Who it’s for

它面向构建生产 AI 推理系统的开发者和工程师，帮助他们避免硬件锁定，并保持单一代码库以实现多平台部署。

Highlights

Multi-Hardware Support: Native compilation for NVIDIA, AMD, Intel, and TPU/Trainium accelerators.
Unified Codebase: Run any model on many hardwares using one codebase.
LLM Support: Out-of-the-box support for Llama 3.1/3.2, Qwen 3.5, and LFM 2.5.
Flexible Loading: Ability to load models from Hugging Face, S3, or local directories via a VFS layer.
High Performance: Compiled directly to hardware for peak execution speed.