Reflection Beam 501B Open-Weight Model: Architecture, Training, and Early Benchmarks

Key Takeaway

Reflection’s Beam model introduces a 501 B‑parameter sparse Mixture‑of‑Experts (MoE) architecture with only 23 B active parameters at inference, delivering coding and agentic capabilities comparable to larger frontier models while using 3–4× less compute per token.

Model Overview

What Beam is – Beam is Reflection’s first open‑weight model, built as a sparse MoE with a total of 501 B parameters but only ~23 B active (expert) parameters per token. The model targets three workloads: coding, reasoning, and agentic (tool‑using) tasks.

Why the architecture matters – By activating a small expert subset per token, Beam reduces the FLOP count required for inference. The authors claim 3–4× lower compute than GLM‑5.2 and even larger efficiency gains versus 2 T+ models such as Qwen‑3.8‑Max.

Pre‑training Foundations

Data volume – 23.8 trillion curated tokens from web sources, proprietary licensed datasets, and extensive code corpora.
Compute budget – Pre‑training completed in under four weeks on 6,144 NVIDIA GB300 NVL72 GPUs, achieving 92.3 % good‑put in the final phase.
Quality‑centric pipeline – Custom classifiers filtered and re‑weighted web, code, and STEM content; ~95 % of raw internet tokens were discarded, yet 1.8 trillion high‑quality tokens that typical pipelines would miss were retained.
MoE stability – Load‑balancing without auxiliary losses, cosine‑decay expert‑bias updates, and sequence‑level balancing yielded near‑uniform expert utilization (busy‑expert load 1.04×). Residual‑stream scaling, SandwichNorm, and FP32 accumulation kept activation norms bounded across 52 layers.

High‑Compute Reinforcement Learning (RL)

Scale – 10.5 K NVIDIA GB300 GPUs ran for four weeks, generating >100 M rollouts (max context 256 K tokens) across ~1 M sandbox environments.
Algorithmic advances – Asynchronous policy‑gradient training with mechanisms to handle policy staleness (samples up to one day old) and numerical mismatch, preserving stability even with large rollout lag.
Efficiency gains – A controllable length‑penalty encouraged token‑efficient solutions early in RL; later, longer reasoning improved performance on harder tasks, giving users a tunable “reasoning effort” parameter.
Generalization – RL training on coding and terminal tasks transferred to unseen domains such as web browsing and OCR, despite those tasks not being present in the RL mixture.

Benchmark Performance

Coding & Agentic Tasks (selected results)

Benchmark	Beam	Inkling	Nemotron 3 Ultra	GLM 5.2	GLM 5.3	Kimi K3	Qwen 3.8 Max	DeepSeek V4.1 Flash
DeepSWE v1.1	44.4	NR	NR	44.0	61.0	68.0	51.0	74.2
SWE Bench Pro v2‑Hard	77.2	56.9	NR	NR	84.3	88.2	NR	NR
Terminal Bench v2.1	80.1	63.8	56.4	81.0	88.2	88.3	86.6	90.6
SWEBench Verified	80.9	77.6	70.7	NR	NR	NR	NR	NR

Interpretation – Beam matches or exceeds GLM‑5.2 on several coding benchmarks while using a fraction of the inference compute. It trails the largest open models (e.g., Kimi K3, DeepSeek V4.1 Flash) on raw scores but compensates with lower FLOPs per token.

Infrastructure for Frontier‑Scale RL

Asynchronous execution – Rollouts continue while new policy checkpoints are published; each token records its originating version.
Dynamic compute allocation – Inference‑to‑training GPU ratios varied between 3.9:1 and 5.4:1; trainer mesh sizes were re‑scaled without losing state.
Fast weight propagation – Median 12 s to distribute new weights across the inference fleet, reducing cross‑rack traffic by 75 % and achieving a 2.2× speed‑up.
Reliability – 71 inference incidents caused only 0.02 % loss of GPU‑minutes; median recovery time was eight minutes.
Sandbox scaling – Up to 170 K concurrent sandboxes; >1 B sandbox creation requests processed across multiple clouds and regions.

Safety & Alignment

Two‑stage alignment – A separate SFT + RL pipeline trained a safety‑focused teacher model; both teachers were merged via multi‑teacher on‑policy distillation.
Alignment principles – (1) Do not break safety rules; (2) Consistently satisfy qualities such as factuality and uncertainty acknowledgment; (3) Adopt a direct, thorough, efficient interaction style.
Reward modeling – Non‑verifiable RL environments used a generative reward model; reward‑prediction correlated with post‑RL improvements (r = 0.79), enabling systematic reduction of hallucinations and over‑formatting.
Deliberative alignment – Iterative adversarial data collection generated harmful prompts; successful attacks were folded back into SFT, improving robustness against jailbreaks.

Community Reception (Hacker News Highlights)

Positive notes – Users praised the addition of another open‑weight contender, emphasizing the importance of competition for the ecosystem.
Critiques – Several commenters pointed out that Beam’s benchmark tables omit stronger open models (e.g., DeepSeek V4.1 Flash) and that the model is not yet publicly downloadable, limiting immediate verification.
Hardware concerns – Questions were raised about the feasibility of running a 501 B MoE model on consumer‑grade hardware; the current access method is via a wait‑list API.
Geopolitical context – Some observers contrasted Western and Chinese open‑weight efforts, noting that Chinese labs currently lead in raw capability.

Outlook

Reflection plans to release Beam’s weights under an Apache 2.0 license later this month, along with a technical report, model card, and developer artifacts. The model is positioned as a “workhorse” for enterprise coding and agentic workloads, with a focus on inference efficiency rather than raw size. Subsequent releases are expected to build on the same MoE and high‑compute RL infrastructure, aiming to narrow the gap with top frontier models.

All figures and benchmark numbers are taken directly from Reflection’s blog post and the associated Hacker News discussion; no external data has been added.