whitecircle/halo

What it solves

Halo is a high-performance training framework designed to make frontier-level training of large language models (LLMs) and multimodal models accessible. It addresses the inefficiency and complexity of distributed training by providing a unified codebase that scales from a single GPU to multi-node clusters while maintaining compatibility with the Hugging Face ecosystem.

How it works

Halo integrates advanced parallelism strategies—including Expert Parallelism (EP), Context Parallelism (CP), Tensor Parallelism (TP), and Expert Tensor Parallelism (ETP)—directly into existing Hugging Face models without requiring separate distributed implementations. It leverages PyTorch primitives (FSDP2, DTensor, DeviceMesh) and specialized kernels like FlashAttention 4, DeepEP, and Liger to maximize throughput. For reinforcement learning (RL), it uses an asynchronous architecture that separates training (via Transformers) from rollouts (via vLLM or SGLang), allowing them to overlap through a prefetch queue.

Who it’s for

It is built for researchers and engineers training large-scale models who need high training throughput, support for Mixture-of-Experts (MoE) and long-context sequences, and a seamless workflow that keeps checkpoints compatible with standard Hugging Face from_pretrained loaders.

Highlights

• Hugging Face Native: Trains models directly and saves checkpoints as standard SafeTensors.
• Extreme Throughput: Delivers up to 2.8× the training throughput of stock TRL on B300 GPUs.
• Flexible Parallelism: Independent configuration of EP, CP, TP, and ETP to optimize for MoE or long-context workloads.
• Asynchronous RL: Supports multi-turn RL with a clean split between the trainer and the rollout server.
• Hardware Optimized: Includes specialized support for Blackwell and Hopper architectures, including FA4 and DeepGEMM.