Cognition SWE-2 release: performance, cost, and training innovations

SWE-2 pushes the Pareto frontier of coding agents

Cognition’s new SWE-2 model attains a 50.0% solve rate on the FrontierCode 1.1 Main benchmark—within one point of the leading Fable 5.1—while being 64 % cheaper to run. The result demonstrates that a single model can dominate the cost‑performance trade‑off curve across multiple effort levels.

Key performance numbers (higher is better unless noted)

Benchmark SWE‑2 Kimi K3 Grok 4.6 Fable 5.1 GPT‑5.6 Sol GPT‑6 Astra SWE‑1.7
FrontierCode 1.1 Main 50.0 % 44.2 % 48.0 % 50.9 % 47.5 % 53.3 % 42.0 %
DeepSWE 1.1 73.0 % 68.5 % 67.5 % 67.4 % 72.7 % 74.1 % 37.7 %
Terminal‑Bench 2.1 92.8 % 88.3 % 88.4 % 91.4 % 88.8 % 89.9 % 81.5 %
Terminal‑Bench 4 27.3 % 21.5 % 20.3 % 55.8 % 37.3 % 57.9 % 7.6 %

SWE‑2 outperforms its predecessor SWE‑1.7 on every benchmark and beats Grok 4.6 on both score and cost. It matches GPT‑5.6 Sol and Fable 5/5.1 at a fraction of their price and reaches within a few points of GPT‑6 Astra while costing only a quarter of the latter.

How SWE‑2 achieves higher intelligence and lower cost

1. Pareto‑informed cost penalties in a single RL run

  • Linear cost penalty – The reward function is R = S – λₑ·C, where S is binary success, C is rollout cost, and λₑ is tuned to the local slope of the base model’s cost‑performance curve for effort level e.
  • Why linear? – A proof (Appendix B) shows that only an affine penalty guarantees that the expected reward depends solely on average cost and solve rate, regardless of the rollout distribution.
  • Slope matching – Setting λₑ equal to the frontier’s slope makes the iso‑reward line tangent to the Pareto curve, ensuring that any increase in reward also pushes the frontier outward.

“Letting λₑ = m (the local slope) ensures the objective is unaffected by movements along the Pareto curve, so improvements directly shift the frontier upward.” – Cognition blog

2. Length‑weighted reward baseline

  • The baseline b̂ = Σ Rᵢ·Lᵢ / Σ Lᵢ (reward weighted by rollout length) approximates the optimal variance‑reducing baseline without extra backward passes.
  • Empirical ablations show a significant reduction in KL divergence between inference and training policies, stabilizing RL.

3. RL rollout serving and speculative decoding

  • DSpark speculative decoding – A draft model proposes tokens that the policy model verifies, increasing throughput by 10‑20 %.
  • SpecForge‑trained draft – Extends accepted token lengths by ~15 % and is updated online to track the evolving policy.
  • Low‑precision MoE inference – NVFP4/FP8 kernels and quantization‑aware training keep memory usage low while preserving KL alignment.

4. Data scale and verifier hardening

  • Tripled the number of RL environments and expanded repository sources.
  • Added instruction‑following overlays to keep models on task while handling multiple simultaneous instructions.
  • Implemented a recursive flywheel: rollouts from intermediate checkpoints are used to discover and patch verifier failures, reducing reward‑hacking.

Behavioral improvements observed in SWE‑2

  • Focused exploration – Median first edit occurs after 18 steps (vs. 48 for SWE‑1.7), cutting exploration overhead by 62 %.
  • Test coverage – Generates more comprehensive end‑to‑end tests, catching regressions earlier.
  • Resourcefulness – When a required integration is unavailable, SWE‑2 can reconstruct needed data from existing context (e.g., Slack history).
  • Verification discipline – Re‑derives conclusions when challenged, runs evidence‑gathering artifacts, and avoids blind agreement.
  • Effort‑level differentiation – Medium effort excels on simple/intermediate tasks with low cost; high and max effort allocate more planning and verification for complex problems.

Trustworthiness evaluation

Cognition re‑ran its internal propaganda/censorship and context‑dependent vulnerability tests on six models (including SWE‑2). Results:

  • Propaganda & censorship – SWE‑2 passes 98.0 % overall (99.8 % English, 95.2 % Simplified Chinese, 99.1 % Traditional Chinese).
  • Context‑dependent vulnerability – No statistically significant framing effect for any model; SWE‑2’s mean vulnerability shift is near zero, matching peers.

Community reaction on Hacker News

  • Skepticism about benchmark generalisation – A top comment notes the large gap between Terminal‑Bench 2.1 (92.8 %) and Terminal‑Bench 4 (27.3 %) as evidence of possible “bench‑maxing”.
  • Concerns over post‑training from Kimi K3 – Some users question whether improvements are merely inherited from the strong K3 base rather than novel engineering.
  • Open‑weight vs. closed‑weight debate – Several commenters ask whether SWE‑2 will be released with open weights, comparing it to models like DeepSeek Flash 4.1.
  • Product experience – Mixed reports: a few users found previous SWE‑1.7 unusable, while others report that Devin (the UI for SWE‑2) improves productivity on pull‑request automation.
  • Availability – The model is currently accessible through Cognition’s Devin Desktop, CLI, Web, and Fusion platforms; no open‑router or third‑party API integration is announced.

What the release means for the coding‑agent landscape

  • Cost‑performance dominance – SWE‑2 demonstrates that a single, multi‑effort RL‑trained model can simultaneously achieve top‑tier solve rates and substantially lower per‑task cost.
  • Methodological contribution – The Pareto‑informed linear cost penalty provides a principled, reproducible recipe for training models that serve a spectrum of user budgets.
  • Potential for broader adoption – If Cognition opens the weights or offers competitive API pricing, SWE‑2 could become a viable alternative to proprietary agents such as Fable 5.1 or GPT‑6 Astra.
  • Open questions – The community remains uncertain about real‑world generalisation beyond the presented benchmarks and about the openness of the model.

References

  1. Lu et al., FrontierCode 1.1, July 2026. https://cognition.com/blog/frontier-code-1.1
  2. Pan et al., SWE‑1.7: Frontier Intelligence at a Fraction of the Cost, July 2026. https://cognition.com/blog/swe-1-7
  3. Kimi Team, Kimi K3: Open Frontier Intelligence, arXiv:2607.24653, July 2026. https://arxiv.org/abs/2607.24653
  4. Kool et al., Buy 4 REINFORCE Samples, Get a Baseline for Free!, ICLR 2019 workshop. https://openreview.net/pdf?id=r1lgTGL5DE
  5. Greensmith et al., Variance Reduction Techniques for Gradient Estimates in RL, JMLR 5, 2004. https://jmlr.org/papers/volume5/greensmith04a/greensmith04a.pdf
  6. Hao et al., On‑Policy RL with Optimal Reward Baseline, arXiv:2505.23585, May 2025. https://arxiv.org/abs/2505.23585
  7. Cheng et al., DSpark: Confidence‑Scheduled Speculative Decoding, arXiv:2607.05147, July 2026. https://arxiv.org/abs/2607.05147
  8. Li et al., SpecForge: Efficient Training for Speculative Decoding, arXiv:2603.18567, March 2026. https://arxiv.org/abs/2603.18567
  9. Cognition Team, Measuring the Trustworthiness of Open‑Source‑Derived Models, July 2026. https://cognition.com/blog/measuring-open-source-model-trustworthiness

Sources

Related