Kimi K3 Release and Lessons from the Pelican SVG Benchmark

Kimi K3’s debut and why it matters

Moonshot AI announced Kimi K3 on 16 July 2026 as their most capable model to date, with 2.8 trillion parameters and a promised open‑weight release by 27 July 2026. The model is priced at $3 per million input tokens and $15 per million output tokens, matching Anthropic’s Claude Sonnet pricing and making it the most expensive Chinese‑lab model so far. Its self‑reported benchmarks place K3 ahead of Claude Opus 4.8 max and GPT‑5.5 high, while trailing only Claude Fable 5 and GPT‑5.6 Sol.

“On our private long‑horizon knowledge work evaluation, Kimi K3 reaches an overall Elo of 1547, +732 points from K2.6 and behind only Claude Fable 5.” – Artificial Analysis report

“Cost per task ($0.94) is similar to GPT‑5.6 Sol ($1.04), ~½ the price of Opus 4.8 ($1.80) and higher than open‑weights peers.” – same report

These figures show that K3 delivers a competitive quality‑to‑price ratio while moving into the 3‑trillion‑parameter class previously claimed only by DeepSeek’s 1.6 T v4 Pro.

The pelican‑on‑a‑bicycle SVG test: a quick sanity check

Simon Willison used the OpenRouter proxy and his llm-openrouter CLI to ask K3 to “Generate an SVG of a pelican riding a bicycle.” The request produced a 95‑token prompt and 16,658 output tokens (13,241 reasoning tokens), costing about $0.25. When the same SVG image was fed back to K3 via its vision capability, the model returned a concise alt‑text description for $0.006.

What the numbers reveal

Reasoning effort matters – K3 currently offers a single “max” reasoning mode, which consumed 13 k reasoning tokens for a 3.4 k token response, making the pelican task relatively expensive.
Hidden system prompt – Prompting K3 with a trivial "hi" counted 86 tokens, suggesting an ~85‑token hidden system prompt that the model refuses to reveal. This mirrors observations from DeepSeek‑V4’s max‑mode prompt.
Vision works – The alt‑text generated from the SVG is accurate and detailed, confirming that K3’s multimodal pipeline can interpret its own graphic output.

Why the pelican benchmark still matters

The pelican SVG prompt has been a "hello world" for LLMs for over 21 months. It originally correlated surprisingly well with overall model quality, but recent releases (GPT‑5.6, Claude Fable 5, GLM‑5.2) have outpaced it, exposing its limits.

“The biggest limitation of the pelican is that it doesn’t touch at all on the thing that matters most for today’s model: agentic tool calling and the ability to operate tools reliably as conversations grow in length.” – Simon Willison

Practical takeaways for practitioners

Cost‑per‑task estimation – The pelican test gives a rough upper bound on token usage and monetary cost for a moderately complex generation task.
Model‑agnostic sanity check – Successfully receiving a valid SVG confirms that a model can handle structured text generation and basic geometry.
Version‑to‑version comparison – Running the same prompt across model families (e.g., K2.6 → K3) highlights incremental improvements in reasoning efficiency and output quality.
Prompt token accounting – Observed hidden tokens remind developers to factor in system prompts when budgeting API usage.

Community insights and criticisms

Parameter count vs. attention density – Some commenters note that GLM‑5.2, despite being smaller, outperforms larger rivals, suggesting that attention mechanisms may matter more than raw parameter count.
Training data leakage concerns – A few participants speculate that pelican‑on‑a‑bicycle SVGs may already exist in training corpora, potentially inflating benchmark scores.
Benchmark stability – Others argue that a static test like the pelican prompt provides historical stability, while evolving benchmarks capture current capabilities better.
Tool‑calling gap – Multiple comments emphasize that modern LLM evaluation should focus on tool use, long‑context reasoning, and agentic behavior rather than isolated image generation.

How to run the pelican test yourself

Install Simon Willison’s LLM CLI (pip install llm).

Use OpenRouter (or a direct API key) to call the model:

llm -m openrouter/moonshotai/kimi-k3 'Generate an SVG of a pelican riding a bicycle'

For vision, feed the resulting SVG back to the model with an alt‑text prompt.
Track token usage via the response metadata or a service like llm‑prices.com.

Bottom line

Kimi K3 marks a significant step for Chinese LLMs, offering competitive quality at a price comparable to leading U.S. models. The pelican‑on‑a‑bicycle benchmark, while no longer a definitive quality proxy, remains a useful low‑overhead sanity check for cost, reasoning effort, and multimodal capability. As LLMs mature, developers should supplement such static prompts with evaluations that stress tool use, long‑context coherence, and real‑world agentic performance.