Project-HAMi/HAMi

What it solves

HAMi addresses the inefficiency of GPU allocation in Kubernetes clusters, where whole GPUs are often assigned to small jobs, leading to wasted resources. It also solves the problem of managing diverse accelerator hardware (NVIDIA GPUs, NPUs, DCUs, etc.) through a single, unified workflow, preventing vendor lock-in and simplifying infrastructure management for AI teams.

How it works

HAMi acts as a virtualization middleware layer between Kubernetes and the hardware. It uses a mutating webhook, a scheduler extender, and device plugins to intercept pod submissions and allocate specific fractions of an accelerator's memory or compute cores. This allows multiple workloads to share a single physical device while maintaining resource isolation, all without requiring changes to the application code.

Who it’s for

Platform engineers and AI infrastructure teams managing Kubernetes-based AI clusters who need to increase GPU utilization, support multi-tenant environments (like notebooks and inference servers), and manage heterogeneous hardware from various vendors.

Highlights

• Device Sharing: Allocate fractions of a physical accelerator based on memory, cores, or device count.
• Heterogeneous Support: Unified management for NVIDIA GPUs and various other accelerators (NPUs, DCUs, MLUs, etc.).
• Device-Aware Scheduling: Supports binpack, spread, and topology-aware policies to optimize workload placement.
• Zero Application Changes: Works with standard Kubernetes resource requests and limits.
• Observability: Includes a WebUI and Grafana dashboards for monitoring accelerator usage.