microsoft/pai

Resource scheduling and cluster management for AI

What it solves

OpenPAI provides a full-stack platform for organizations to share and manage powerful AI computing resources (such as GPU and FPGA farms) and common AI assets like models, data, and environments. It simplifies IT operations for AI by providing a centralized place to run complete training pipelines.

How it works

OpenPAI is a modular platform built on Kubernetes that decouples computing hardware from software using Docker technology. This allows users to run distributed jobs and switch between different deep learning frameworks in consistent environments. It includes several key components:

  • Job Orchestration & Scheduling: Uses a Framework Controller and a specialized Kubernetes Scheduler Extender (HiveD) for multi-tenant GPU clusters.
  • Management Interfaces: Offers a Web Portal, a command-line tool (paictl), and a VS Code extension for job submission and monitoring.
  • Asset Management: Includes a Marketplace for sharing job templates and examples, and integrated storage management for team-based data sharing.
  • Hardware Support: Supports on-premises, hybrid, and public cloud deployments with support for heterogeneous hardware including CPUs, GPUs, FPGAs, and InfiniBand.

Who it’s for

  • Cluster Administrators: IT professionals responsible for the deployment, availability, and maintenance of computing resources.
  • Cluster Users: Machine learning and deep learning researchers, data scientists, students, and teachers who need to train and serve AI tasks.

Highlights

  • Heterogeneous Hardware Support: Native support for GPUs, FPGAs, and InfiniBand.
  • Full-Stack Solution: Covers everything from user authentication and group management to job runtime and error analysis.
  • Kubernetes Integration: Compatible with the Kubernetes ecosystem while providing specialized scheduling for AI workloads.
  • Modular Architecture: Components can be plugged in or customized to suit specific organizational needs.
  • Collaborative Ecosystem: Integrated marketplace for one-click reproduction of shared AI jobs.

Related

  • Dispatch
    OpenAI Scaling Kubernetes to 7,500 Nodes

    OpenAI scaled a single Kubernetes cluster to 7,500 nodes to provide a simple, scalable infrastructure for machine learning research, overcoming challenges in networking, API server load, and monitoring.

  • Project
    hyperspaceai/agi

    A decentralized P2P network where autonomous AI agents collaboratively conduct ML research, pool compute for distributed training and inference, and share discoveries via a global leaderboard.

  • Project
    Project-HAMi/HAMi

    A Kubernetes GPU virtualization and scheduling middleware that enables sharing and isolating accelerator resources across multiple AI workloads to increase hardware utilization.

  • Project
    Haohao-end/openagent

    A full-stack AI agent platform for building, orchestrating, and publishing AI applications with visual workflows, RAG integration, and OpenAPI delivery.

  • Project
    cfregly/ai-performance-engineering

    A comprehensive collection of code and resources for optimizing AI systems, focusing on GPU tuning, distributed training, and high-throughput inference scaling.