maze-agent/Maze

Maze: A Distributed Framework for LLM Agents

What it is – Maze is an open‑source Python framework that lets you turn LLM‑agent programs into durable, observable workflows that run on a cluster. It sits on top of Ray, adding a unified run API, heterogeneous resource queues (GPU, CPU, I/O), built‑in model‑service management, and a visual Workbench for designing and inspecting DAGs.

Why it matters – Large‑language‑model agents often need many heterogeneous steps (prompting, retrieval, fine‑tuning, data I/O) and must share GPUs across many users. Maze coordinates those steps, keeps state across restarts, and gives you a single way to submit a workflow whether you write it in pure Python, via the LangGraph SDK, or as a static maze.yaml spec.

Core concepts

Concept	What it does
Run	The immutable identifier for a workflow execution. All events, logs, artifacts, and placement info are attached to a Run.
Task	A Python function annotated with `@task`. You declare required resources (`cpu_num`, `gpu_mem`, `io_num`).
Workflow	A static DAG built with `@workflow` (or submitted as a JSON contract).
DynamicRun	A mutable run that can have tasks appended at runtime, useful for agents that decide the next step on the fly.
Workbench	A web UI (React) that provides a drag‑and‑drop DAG editor, run‑time canvas, and cluster‑resource monitor.
Scheduler	Handles three independent queues (GPU, CPU, I/O). Supports simple FIFO (`FCFS`) or the research‑grade HACS algorithm that re‑prioritises tasks based on historic runtimes.
Model routing	Maze discovers local checkpoints (vLLM or Transformers), spins up a model server on demand, and puts tasks that need the model into a Model Wait state so they don’t block GPU queues.

Highlighted features

Visual workflow authoring – Drag‑and‑drop DAG editor, reusable task catalog, validation, and one‑click submission.
Heterogeneous scheduling – Separate GPU, CPU, and I/O queues; pluggable FCFS or the paper‑aligned HACS algorithm with tunable parameters.
Static + dynamic DAGs – Write a fixed DAG with @workflow or grow a run at runtime via DynamicRun.append_task.
Distributed model serving – Automatic discovery and deployment of local checkpoints (vLLM/Transformers), explicit model‑wait state, LRU scale‑in of idle instances.
Durable state – Runs survive process crashes; task state, retries, timeouts, and content‑addressed artifacts are persisted.
Unified API surface – Same Python SDK, LangGraph adapter, and Workbench UI all talk to the Core through the maze.workflow/v1 contract.
Observability – Structured events, logs, artifact URLs (maze://artifacts/sha256/<hash>), and a canvas view that shows live task status.
Cluster management – CLI commands to list resources, queues, join workers, and reconcile node registration.

Typical use cases

LLM‑agent pipelines – Combine prompting, retrieval, tool‑calling, and fine‑tuning steps that need both CPU and GPU resources.
Training jobs with auxiliary I/O – Run data‑preprocessing tasks on the I/O queue while GPUs train models, preventing I/O from blocking GPU dispatch.
Research reproducibility – Store every run’s inputs, code, and artifacts in a content‑addressed store; replay runs later with the same workspace.
Multi‑tenant clusters – Separate queues ensure a long‑running CPU job can’t starve a short GPU inference request.
Dynamic agent reasoning – Use DynamicRun to let an LLM decide the next action at runtime, appending new tasks on the fly.

Quick start (from the README)

# Install
pip install maze-agent

# Start the head node and the web UI (Workbench)
maze start --head --port 8000 --playground --detach
# UI → http://localhost:5173   API → http://localhost:8000

# Add a worker that registers with the head
maze start --worker \
  --addr HEAD_IP:8000 \
  --agent \
  --heartbeat-interval 20

Define a simple static workflow

from maze import MaClient, task, workflow

@task(resources={"cpu_num":1, "gpu_mem":0, "io_num":0})
def greet(text: str):
    return {"result": f"Hello {text}"}

@task(resources={"cpu_num":1, "gpu_mem":0, "io_num":0})
def uppercase(result: str):
    return {"upper": result.upper()}

@workflow
def hello(name: str):
    g = greet(name)
    return uppercase(g.result)

client = MaClient("http://localhost:8000")
run = client.create_workflow_from(hello, inputs={"name":"Maze"}).run()
print(client.wait_run(run)["result_summary"])

Dynamic workflow example

from maze import MaClient, task

@task(resources={"cpu_num":1})
def summarize(topic: str=""):
    return {"summary": f"Maze can build workflows dynamically for {topic}."}

client = MaClient("http://localhost:8000")
run = client.create_dynamic_run(max_tasks=10)
task_ref = run.append_task(summarize, inputs={"topic":"agent runtime"})
run.wait_for_task(task_ref)
run.finalize({"status":"done"})
print(run.status())

Running a full application spec

Create maze.yaml:

name: gpu-demo
command: python train.py
workspace: .
resources:
  cpu_num: 4
  gpu_mem: 8192
  io_num: 0
env:
  conda: maze
  vars:
    DATASET: sample
artifacts:
  - outputs/
timeout_seconds: 1800
retries:
  max: 1
  backoff_seconds: 5
  on: [node_lost, resource_unavailable]

Then:

maze app validate maze.yaml
maze run maze.yaml --wait
maze runs logs <run_id>

All execution metadata appears in the unified Run history.

Architecture at a glance

Python SDK / LangGraph / Workbench / App Spec
        |
   maze.workflow/v1 (JSON contract)
        |
      Maze Core – runs, events, logs, artifacts
        |
   Scheduler + Ray – heterogeneous queues, placement
        |
   Head & Worker nodes – actual task execution, model services

The Core owns the Run ID; workers only execute tasks and report back.

Community & citation

If you use Maze in research, cite:

@inproceedings{gu2026maze,
  title={Maze: A Distributed Framework for Large Language Model Agents},
  author={Jing Gu and Zhuang Xing and Yiheng Yang and ...},
  booktitle={Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis},
  year={2026}
}

The project is maintained by contributors from Huazhong University of Science and Technology, Huawei, and other institutions.

Where to learn more

Website: https://mazeagent.net/
Docs: https://maze-doc-new.readthedocs.io/en/latest/
GitHub: https://github.com/maze-agent/Maze

Bottom line – Maze gives you a production‑grade runtime for LLM‑agent pipelines: it handles scheduling across heterogeneous resources, keeps runs durable, serves models on demand, and provides a visual UI for design and debugging, all through a single Python‑friendly API.