kruize/autotune

Autonomous Performance Tuning for Kubernetes!

What it solves

Kruize addresses the challenge of manually tuning Kubernetes resource allocations. It prevents over-provisioning, which wastes money, and under-provisioning, which degrades application performance, by automatically suggesting the correct amount of CPU, memory, and GPU resources needed for workloads.

How it works

Kruize connects to a monitoring stack (like Prometheus or Thanos) to analyze historical resource usage patterns. It uses metadata and metric profiles to understand the environment and define what constitutes "good performance." Based on this analysis, it generates right-sizing recommendations for:

  • Containers: CPU and memory requests and limits.
  • Namespaces: Resource quota limits.
  • NVIDIA GPUs: Optimal MIG slice configurations for A100 and H100 accelerators.
  • Application Runtimes: Optimized configurations for JVMs and frameworks like Quarkus (Alpha).

Users can choose between cost-optimized or performance-optimized profiles and set custom observation windows.

Who it’s for

It is designed for Kubernetes and OpenShift administrators and DevOps engineers who want to reduce infrastructure costs and improve workload stability without manual trial-and-error tuning.

Highlights

  • Multi-resource support: Optimizes CPU, memory, and GPU (MIG slices).
  • GPU Optimization: Specifically supports NVIDIA A100 and H100 accelerators.
  • Integration: Works with Prometheus and Thanos for data collection.
  • Flexible Profiles: Offers predefined and custom terms for observation windows and optimization goals (cost vs. performance).
  • Deployment: Available via a dedicated operator for simplified management.

Related

  • Project
    NVIDIA/k8s-device-plugin

    The NVIDIA k8s‑device‑plugin is an official DaemonSet that implements Kubernetes’ device‑plugin API for NVIDIA GPUs. It discovers GPUs, reports health, and makes them schedulable via the `nvidia.com/gpu` resource (with optional MIG and shared‑GPU modes). Install the NVIDIA driver and container toolkit on each node, deploy the plugin (static manifest or Helm), then request GPUs in pod specs. Configuration is possible via flags, env‑vars, or a YAML file, covering MIG strategy, device‑list delivery, shared‑GPU (time‑slicing or MPS), and more. The plugin is beta, lacks full health‑check/cleanup, and only the official NVIDIA version is supported.

  • Project
    kite-org/kite

    Kite is an open-source Kubernetes workspace for multi-cluster operations, observability, and AI-assisted troubleshooting.

  • Project
    Project-HAMi/HAMi

    A Kubernetes-native virtualization middleware that enables sharing and isolating GPU and other AI accelerator resources across multiple workloads to increase hardware utilization.

  • Project
    ome-projects/ome

    A Kubernetes operator for enterprise-grade LLM serving that automates model management, runtime selection, and GPU resource optimization.

  • Project
    triton-inference-server/model_analyzer

    A CLI tool for Triton Inference Server that automates the search for optimal model configurations to balance performance and resource usage on specific hardware.