IBM Granite 4.1 LLMs release notes / technical overview

IBM Granite 4.1 LLMs release notes / technical overview

IBM has released Granite 4.1, a family of dense, decoder-only Large Language Models (LLMs) consisting of 3B, 8B, and 30B parameter variants. These models are trained on approximately 15 trillion tokens and utilize a multi-stage pre-training and reinforcement learning pipeline to achieve high performance in math, coding, and instruction following, with the 8B model notably matching or surpassing larger Mixture-of-Experts (MoE) architectures.

Model Architecture

Granite 4.1 models utilize a decoder-only dense transformer architecture. All three model sizes (3B, 8B, and 30B) share the same training pipeline and data strategy, differing only in their specific architectural dimensions.

Key architectural components include:

  • Grouped Query Attention (GQA)
  • Rotary Position Embeddings (RoPE)
  • SwiGLU activations
  • RMSNorm
  • Shared input/output embeddings
Component 3B Dense 8B Dense 30B Dense
Embedding size 2560 4096 4096
Number of layers 40 40 64
Attention head size 64 128 128
Number of attention heads 40 32 32
Number of KV heads 8 8 8
MLP hidden size 8192 12800 32768
MLP activation SwiGLU SwiGLU SwiGLU
Position embedding RoPE RoPE RoPE

Five-Phase Pre-Training Strategy

The models are trained from scratch on ~15 trillion tokens using a five-phase strategy that progressively shifts from broad web-scale data to highly curated, domain-specific content.

Phase 1: General Pre-Training

This phase establishes foundational language understanding using 10 trillion tokens. The data composition is primarily CommonCrawl (59%), followed by Code (20%), Technical (10.5%), Math (7%), and Multilingual (~2%).

Phase 2: Math and Code Focus

Phase 2 increases reasoning capabilities by increasing the proportion of mathematical and programming data. The mixture includes Math (35%), Code (30%), CommonCrawl-HQ (12%), Technical (10%), Synthetic (9%), and Multilingual (3%).

Phases 3 & 4: High-Quality Data Annealing

These mid-training phases focus on high-quality data mixtures and include the introduction of chain-of-thought and synthetic instruction data.

  • Phase 3 (2T tokens): Blends CommonCrawl-HQ, Math, Code, Technical, Synthetic, Long Chain-of-Thought, and Language/Code Instructions.
  • Phase 4 (0.5T tokens): Further refines the model with a focus on the highest-quality data, heavily weighted toward CommonCrawl-HQ (40%), Code (20%), and Math (20%).

Phase 5: Long Context Training (LCE)

The final phase extends the context window from 4K to 512K tokens through a staged extension process (32K, 128K, and finally 512K). The 512K extension for the 8B and 30B models utilizes a mixture of 80% books and 20% code repository data. To prevent degradation of short-context performance, a model merge is performed after each LCE stage.

Supervised Fine-Tuning (SFT) and Quality Control

Granite 4.1 models undergo SFT using approximately 4.1 million high-quality samples. To ensure data integrity, IBM employs an LLM-as-Judge framework alongside rule-based filtering.

LLM-as-Judge Framework

The framework evaluates assistant responses across six weighted dimensions: instruction following, correctness, completeness, conciseness, naturalness, and calibration. The judge ignores system prompts, user inputs, and retrieved documents to focus strictly on the model's response. Hard-reject rules are applied for critical defects such as hallucinations, false premises, or incorrect computations.

SFT Training Configuration

  • Compute: 16 nodes, 4x GB200 per node
  • Epochs: 3
  • Sequence length: 16,384 tokens
  • Effective batch size: 256 samples/iter

Multi-Stage Reinforcement Learning (RL)

After SFT, the models undergo a multi-stage RL pipeline to optimize specific capabilities while minimizing catastrophic forgetting.

Training Methodology

The models use On-policy GRPO (Group Relative Policy Optimization) with DAPO (Decoupled Clip and Dynamic sAmpling Policy Optimization) loss. The pipeline consists of four sequential stages:

  1. Multi-domain RL: Joint training on math, science, logic, instruction following, structured output, Text2SQL, temporal reasoning, and general chat.
  2. RLHF (Reinforcement Learning from Human Feedback): Uses a multilingual scalar reward model to improve helpfulness and conversational quality.
  3. Identity & Knowledge-Calibration RL: A short stage (~40 steps) to improve self-identification capabilities.
  4. Math RL: A targeted stage to recover and boost mathematical reasoning performance lost during the RLHF stage.

Performance and Benchmarks

Granite 4.1 models demonstrate strong performance across general, math, code, and multilingual tasks. A notable finding is that the Granite 4.1-8B dense model matches or outperforms the previous-generation Granite 4.0-H-Small (32B-A9B MoE) across several key benchmarks, including IFEval, AlpacaEval, MMLU-Pro, and GSM8K.

Key Instruct Model Benchmarks

Benchmark 3B 8B 30B
MMLU (5-shot) 67.02 73.84 80.16
GSM8K (8-shot) 86.88 92.49 94.16
HumanEval (pass@1) 79.27 87.20 89.63
AlpacaEval 2.0 38.57 50.08 56.16
BFCL v3 — 68.27 73.68

Deployment and Infrastructure

Granite 4.1 models are released under the Apache 2.0 license. For efficient inference, FP8 quantized variants are available, which reduce disk footprint and GPU memory usage by approximately 50% compared to 16-bit precision.

Training was conducted on an NVIDIA GB200 NVL72 cluster hosted on CoreWeave, utilizing a 72-GPU NVLink domain for intra-rack communication and a non-blocking Fat-Tree NDR 400 Gb/s InfiniBand network for inter-rack communication.

Sources