meta-pytorch/monarch

Monarch – Distributed Actor Framework for PyTorch

What it is

• A Python‑first library (with a Rust core) that lets you run actors—independent processes that can send messages to each other—on top of PyTorch.
• Actors are organized into meshes (collections) and can communicate via broadcast, point‑to‑point RDMA, or normal RPC calls.
• Built‑in fault‑tolerance: actors form a supervision tree so failures propagate upward and can be recovered automatically.
• Optional tensor engine adds distributed‑tensor sharding and one‑sided GPU/CPU memory transfers using libibverbs (RDMA).

Key concepts

Feature	What it gives you
Remote actors & meshes	Write a class inheriting from Actor, decorate methods with @endpoint, then spawn many copies across GPUs or nodes. Calls return futures you can get().
Supervision trees	Hierarchical process groups automatically restart or clean up failed children, similar to Erlang/Elixir supervision.
RDMA transfers	Low‑latency, zero‑copy moves of GPU/CPU memory between processes when the hardware supports it.
Distributed tensors	Shard a torch.Tensor across actors, letting each work on its slice while the library handles the underlying communication.

Typical workflow (from the README)

from monarch.actor import Actor, endpoint, this_host

# 8 processes, one per GPU
procs = this_host().spawn_procs({"gpus": 8})

class Trainer(Actor):
    @endpoint
    def train(self, step: int):
        ...  # training logic here

trainers = procs.spawn("trainers", Trainer)   # creates 8 Trainer actors
fut = trainers.train.call(step=0)               # broadcast a call
fut.get()                                        # wait for all to finish

Installation

• Pre‑built wheels via pip install torchmonarch (stable) or --pre for nightly builds.
• Source build uses the uv package manager, a nightly Rust toolchain, and optional CUDA/ROCm/RDMA libraries. You can disable the tensor engine (USE_TENSOR_ENGINE=0) for a pure‑actor, CPU‑only install.
• Docker support: build a wheel, then use the provided Dockerfile.nightly to create an image suitable for Kubernetes.

Who it’s for

• Researchers and engineers who need to scale PyTorch training or inference across many GPUs or nodes but want a higher‑level abstraction than raw torch.distributed.
• Teams building fault‑tolerant AI services where individual workers may crash and need automatic recovery.
• Users with access to RDMA‑capable hardware who want ultra‑low‑latency tensor sharding.

Documentation & community

• Hosted docs at https://meta-pytorch.org/monarch/ with a “getting started” tutorial.
• Tests are written in both Rust (via cargo-nextest) and Python (pytest).
• BSD‑3‑Clause license, with third‑party RDMA code under compatible licenses.

Bottom line Monarch is a genuine, production‑grade framework that extends PyTorch with actor‑style distributed programming, fault tolerance, and optional high‑performance RDMA tensor handling. It’s aimed at advanced ML workloads that need more structure and resilience than raw torch.distributed provides.