Anthropic Research: Measuring LLM Exploit Development Capabilities

TL;DR

Anthropic has evaluated the exploit development capabilities of Claude Mythos Preview, finding it capable of constructing complete end-to-end attack chains on widely used software. This represents a significant leap over previous frontier models, suggesting that the specialized expertise required for exploit development will decrease as these capabilities become more common.

V8 Engine Exploitation via ExploitBench

Claude Mythos Preview is the only tested model capable of reliably escaping the V8 sandbox to achieve arbitrary code execution (ACE). Using the ExploitBench framework—which measures 16 distinct capabilities across five tiers (Coverage, Reproduction, Target primitives, Generic primitives, and Full Control)—Mythos Preview achieved ACE on 21 out of 41 tested CVEs.

Key findings from the V8 evaluation include:

• Sandbox Escape: While most models can trigger vulnerabilities, Mythos Preview is the only one that consistently moves from T3 (sandbox primitives) to T2 (generic primitives/sandbox escape).
• Stability and Novelty: In one instance (CVE-2023-6702), Mythos Preview created a near-deterministic exploit where existing public exploits were probabilistic. One of the benchmark authors noted that the model executed a complex exploit plan that human researchers had previously dismissed due to its complexity.
• Performance Gap: No other model achieved a single ACE in the baseline or nudged variants of the benchmark without using a proprietary scaffold.

Broad Target Exploitation via ExploitGym

In evaluations using ExploitGym, which covers 898 patched vulnerabilities across OSS-Fuzz, the V8 engine, and the Linux kernel, Mythos Preview demonstrated superior success rates in achieving unauthorized code execution.

• Success Rates: Mythos Preview successfully achieved unauthorized code execution using the intended vulnerability on 157 tasks, totaling 226 successful flag captures when including alternative vulnerability paths.
• Comparison: In contrast, Claude Opus 4.6 achieved only 15 successes with the intended vulnerability and 36 total flag captures.
• Kernel Exploits: Mythos Preview is one of only two reported models capable of frequently developing kernel exploits.

Smart Contract Exploitation via SCONE-bench

Using an updated SCONE-bench dataset featuring 12 exploits reported after January 1, 2026, Mythos Preview demonstrated a high capacity for financial theft in simulated environments.

• Financial Impact: Mythos Preview exploited $35 million worth of smart contracts, approximately 75% more than the next-closest model.
• Success Rate: Mythos Preview was the only model to successfully exploit every vulnerability tested in the benchmark.
• Growth Trend: The doubling time for model performance in smart contract exploitation has accelerated from 1.1 months (prior to Opus 4.5) to 0.7 months for more recent models.

Implications for AI Safety and Deployment

Anthropic concludes that the ability to develop full end-to-end exploits on global infrastructure is a critical capability that necessitates rigorous, expert-led benchmarking. To mitigate these risks, Anthropic has implemented several measures:

• Project Glasswing: A controlled rollout strategy for Mythos Preview to prevent general release of high-risk capabilities.
• Cyber Verification Program: A system to block potentially malicious cyber threats while maintaining access for legitimate security defenders.
• External Researcher Access Program: An initiative to support the development of high-quality, rigorous evaluations in the cyber domain.