WaveSpeedAI/waverless

What it solves

Waverless is a high-performance serverless GPU task orchestration system designed to manage the execution of AI workloads across various compute providers. It solves the problem of efficiently distributing tasks to GPU workers, ensuring load balancing and resource optimization through smart autoscaling.

How it works

The system uses a pull-based architecture where workers actively request tasks from a central API server. This server manages a task queue, worker management, and an autoscaler that monitors queue depth and resource needs. It supports multiple backends for worker deployment, including Kubernetes, Docker, and Novita Serverless, and integrates with Redis and MySQL for state management.

Who it’s for

It is built for developers and organizations deploying AI models that require scalable, serverless GPU infrastructure to handle fluctuating demand without manual intervention.

Highlights

• Pull-based Architecture: Improves load balancing by allowing workers to pull tasks rather than having them pushed to them.
• Multi-Provider Support: Compatible with Kubernetes, Docker, and Novita Serverless.
• RunPod Compatibility: Allows for zero-code migration from the runpod-python SDK.
• Smart Autoscaling: Scales based on priority, resource awareness, and queue depth.
• Graceful Shutdown: Ensures no tasks are lost during rolling updates.