meta-pytorch/monarch

What it solves

Monarch simplifies the complexity of distributed programming in PyTorch. It addresses the challenges of managing remote processes, coordinating communication between them, and handling hardware-level memory transfers and failures in large-scale AI training and inference workloads.

How it works

Monarch uses a scalable actor-based messaging system where actors are organized into "meshes" for efficient broadcasting. It employs supervision trees to manage fault tolerance, ensuring that failures propagate upward for structured recovery. For high-performance data movement, it integrates point-to-point RDMA transfers via libibverbs and provides a distributed tensor API that allows actors to work with tensors sharded across multiple processes.

Who it’s for

It is designed for developers building large-scale distributed PyTorch applications who need robust fault tolerance, high-performance GPU/CPU memory transfers, and a scalable way to manage remote actors.

Highlights

• Scalable Actor Messaging: Group actors into meshes to broadcast messages efficiently.
• Fault Tolerance: Uses supervision trees to provide structured error behavior and fine-grained recovery.
• RDMA Support: Enables cheap registration and one-sided transfers of GPU or CPU memory.
• Distributed Tensors: Native support for tensors sharded across different processes.
• Flexible Build: Supports CUDA, ROCm, and CPU-only environments.