maze-agent/Maze

What it solves

Maze is a distributed framework designed to turn agent programs into observable, scalable workflows. It solves the problem of managing complex LLM agent tasks across heterogeneous compute resources (CPU, GPU, and I/O) while providing a unified API for execution, recovery, and model serving.

How it works

Maze uses a centralized Core to manage "Runs" and a scheduler that distributes tasks across a cluster of worker nodes using Ray. It supports both static DAGs (defined via @workflow or specifications) and dynamic workflows where tasks are appended at runtime.

Key technical components include:

• Heterogeneous Scheduling: Separate queues for GPU, CPU, and I/O to prevent resource blocking, utilizing either FCFS or the HACS scheduling algorithm.
• Distributed Model Execution: Automatically discovers local checkpoints and deploys vLLM or Transformers instances on demand, routing tasks to available models without blocking the resource queues.
• Durable State: Persists task state, logs, and content-addressed artifacts across process restarts to ensure fault tolerance.
• Unified Interface: Provides a Python SDK, a visual Workbench (DAG editor), and a CLI that all interact with the same Core API.

Who it’s for

It is built for developers and researchers building complex LLM-based agents that require distributed execution, visual workflow orchestration, and efficient GPU resource management.

Highlights

• Visual Workflow Development: A Workbench featuring a DAG editor, task catalogs, and cluster monitoring.
• Resource-Aware Scheduling: Specialized queues for different hardware resources to optimize throughput.
• Dynamic & Static Workflows: Support for both predefined DAGs and runtime-appended tasks.
• Automated Model Lifecycle: On-demand deployment and scaling of model instances (vLLM/Transformers) with LRU scale-in.
• Fault Tolerance: Built-in retries, timeouts, and durable artifact storage using content-addressed references.