kuberay-project/kuberay

KubeRay – Kubernetes Operator for Ray

What it is

• An open‑source CNCF sandbox project that provides a Kubernetes operator to run Ray workloads (distributed training, inference, serving, etc.) on a Kubernetes cluster.
• It defines three custom resources: RayCluster, RayJob, and RayService, letting you declaratively create, scale, and manage Ray clusters directly from Kubernetes manifests.

Key components

Component	Purpose
RayCluster	Manages the full lifecycle of a Ray cluster – creation, autoscaling, fault‑tolerance, and deletion.
RayJob	Spins up a temporary RayCluster, runs a job, and optionally tears the cluster down when the job finishes.
RayService	Combines a RayCluster with a Ray Serve deployment graph, offering zero‑downtime upgrades and high‑availability serving.
kubectl‑ray plugin (beta)	CLI helper (kubectl ray …) that simplifies common KubeRay actions for users less familiar with raw Kubernetes objects.
KubeRay APIServer (alpha)	A thin API layer that can be used by custom UIs or other tooling to manage KubeRay resources.
KubeRay Dashboard (experimental)	Web UI for visualising and managing Ray clusters, jobs, and services.

Why use it

• Declarative: Define Ray workloads in YAML just like any other Kubernetes app.
• Native scaling: Leverages Kubernetes autoscaling mechanisms and integrates with queueing systems (Volcano, Kueue, etc.) for elastic resource allocation.
• Observability: Works with Prometheus, Grafana, and profiling tools (py‑spy) out of the box.
• Production‑ready patterns: Supports zero‑downtime upgrades, high‑availability serving, and automated cleanup of temporary clusters.

Typical use cases

• Distributed model training at scale (e.g., Ray Train).
• Large‑language‑model (LLM) serving with Ray Serve, including GPU/TPU‑backed inference.
• Batch inference pipelines and ensemble model serving.
• Any Ray‑based workload that benefits from Kubernetes’ scheduling, security, and multi‑tenant capabilities.

Getting started

1. Install the operator (e.g., via Helm or kubectl apply of the provided manifests).
2. Choose a custom resource:
   • RayCluster for a long‑running cluster.
   • RayJob for one‑off jobs.
   • RayService for serving APIs with automatic roll‑outs.
3. Apply the corresponding YAML manifest; the operator creates the Ray head and worker pods, handles autoscaling, and monitors health.
4. Optionally install the kubectl ray plugin to run shortcuts like kubectl ray submit ….

Ecosystem integration

• Works with common Kubernetes add‑ons: Ingress controllers (NGINX), observability stacks (Prometheus/Grafana), and batch schedulers (Volcano, Kueue, YuniKorn).
• Documentation and examples are hosted on the Ray docs site, covering training, LLM inference, batch jobs, and more.

Community & support

• Active Slack channel #kuberay and bi‑weekly community meetings.
• Governance and contribution guidelines are provided (GOVERNANCE.md, CONTRIBUTING.md).
• Security issues are reported via the project’s security policy.

License

• Apache‑2.0.

Bottom line KubeRay turns Ray’s powerful distributed compute model into a first‑class Kubernetes workload, letting AI/ML teams deploy, scale, and operate training and serving pipelines using familiar Kubernetes tooling.