WeiboAI/VibeThinker

Tiny Model, Big Logic: Diversity-Driven Optimization Elicits Large-Model Reasoning Ability in VibeThinker-1.5B

What it solves

VibeThinker provides high-performance reasoning capabilities in extremely small model sizes (1.5B and 3B parameters). It challenges the idea that robust reasoning requires massive models by delivering frontier-level performance in mathematics, competitive programming, and STEM reasoning while remaining computationally efficient and cost-effective to train.

How it works

The project uses a post-training methodology called the "Spectrum-to-Signal Principle (SSP)". For the 1.5B model, this involves a two-stage diversity-exploring distillation to generate a wide range of solutions, followed by MaxEnt-Guided Policy Optimization (MGPO) to amplify correct signals. The 3B model builds on this with an upgraded pipeline including curriculum-based supervised fine-tuning, multi-domain reinforcement learning, and offline self-distillation. Additionally, the 3B model introduces Claim-Level Reliability Assessment (CLR) to scale performance during inference time.

Who it’s for

It is designed for developers and researchers who need powerful reasoning models that can run on limited hardware or who are focused on tasks with verifiable answers, such as competitive math and coding.

Highlights

  • Extreme Efficiency: The 1.5B model outperforms significantly larger models (like DeepSeek R1) on specific math benchmarks (AIME24, AIME25, HMMT25).
  • Low Training Cost: The 1.5B model was developed for approximately $7,800, a fraction of the cost of other state-of-the-art reasoning models.
  • Verifiable Reasoning: Specialized for tasks where target answers can be explicitly verified, including STEM and programming.
  • Inference Scaling: Uses CLR to further boost accuracy on math benchmarks during the testing phase.

Related