Kimi K3 Release: 2.8 Trillion Parameter Model and the Pelican Benchmark

Kimi K3 Release: 2.8 Trillion Parameter Model and the Pelican Benchmark

Kimi K3: A New 3T-Class Frontier Model

Moonshot AI has announced Kimi K3, described as the lab's most capable model to date with 2.8 trillion parameters. This release marks the first "open 3T-class model," surpassing the parameter count of DeepSeek's 1.6T v4 Pro. While currently available via website and API, Moonshot AI has promised an open-weight release by July 27, 2026.

Performance and Benchmarks

Kimi K3 demonstrates high competitiveness against global frontier models, particularly in coding and long-horizon tasks:

  • Frontend Code: K3 is currently the leading model on Arena.ai's Frontend Code arena, surpassing Claude Fable 5.
  • Long-Horizon Knowledge Work: According to an Artificial Analysis report, K3 achieved an Elo of 1547, a significant jump of 732 points over Kimi K2.6, placing it behind only Claude Fable 5.
  • General Benchmarks: Self-reported data indicates K3 generally outperforms Claude Opus 4.8 max and GPT-5.5 high, though it trails behind Claude Fable 5 and GPT-5.6 Sol.

Pricing and Efficiency

Kimi K3 is the most expensive model released by a Chinese AI lab to date, priced at $3 per million input tokens and $15 per million output tokens. This aligns it with the pricing of Anthropic's Claude Sonnet series, representing a substantial increase from Kimi K2.6 ($0.95/$4).

Efficiency gains are also evident in token usage; the Artificial Analysis Intelligence Index shows K3 uses 21% fewer output tokens than its predecessor, K2.6.

Analyzing Model Characteristics via SVG Generation

To evaluate the model's practical capabilities and cost, Kimi K3 was tested using a "pelican benchmark"—the generation of an SVG of a pelican riding a bicycle. This exercise reveals several technical characteristics of the model:

Reasoning and Token Costs

Kimi K3 currently utilizes a single "max" reasoning effort level. In the pelican test, the model consumed 13,241 reasoning tokens to produce 3,417 tokens of response. For a single prompt, this resulted in a total cost of 25 cents.

Hidden System Prompts

Testing suggests the presence of a hidden system prompt. A simple prompt of "hi" resulted in 86 tokens, while the prompt "Generate an SVG of a pelican riding a bicycle" totaled 95 tokens. This suggests an approximately 85-token system prompt that the model refuses to leak.

Multimodal Capabilities

K3's vision capabilities are strong. When fed the rendered SVG of the pelican, the model generated a highly accurate and detailed alt-text description, correctly identifying the pelican, the red scarf, the red bicycle, and background elements like the yellow sun and green grass.

The Validity of the "Pelican Benchmark"

While the pelican test serves as a useful "hello world" for new models, its utility as a comparative benchmark for overall intelligence has diminished over time.

Limitations of SVG Benchmarking

  • Lack of Agentic Evaluation: The test does not measure agentic tool calling or the ability to maintain reliability over long conversations, which are critical for modern frontier models.
  • Decoupling from General Intelligence: There is a growing disconnection between a model's ability to generate a specific SVG and its overall capability. For example, GLM-5.2 outperforms GPT-5.6 and Claude Fable 5 in this specific task, despite not being considered a "Fable-class" model.
  • Training Data Contamination: There is significant debate regarding whether these specific prompts have entered the training corpus. Given the popularity of these tests on blogs and GitHub, it is highly probable that models are being trained on the results of previous pelican benchmarks.

Value of the Exercise

Despite its flaws, the pelican test remains valuable for:

  1. Verification: Confirming the model is accessible and the API is functional.
  2. Cost Estimation: Providing a rough baseline for reasoning and output costs.
  3. Spatial Awareness: Confirming the model can output valid SVG code and possesses basic geometric understanding.
  4. Family Comparison: Tracking improvements within a single model family (e.g., the improvement from Kimi 2.5 to K3).

Sources