Mistral AI Leanstral Release

Mistral AI has released Leanstral, an open-source code agent specifically designed for the Lean 4 proof assistant. Leanstral aims to reduce the human review bottleneck in high-stakes software and mathematical research by enabling agents to formally prove implementations against strict specifications.

Leanstral Technical Architecture and Access

Leanstral is a highly efficient model utilizing a sparse architecture with 6B active parameters. It is optimized for proof engineering tasks and leverages parallel inference with Lean acting as a perfect verifier to maintain cost-efficiency and performance.

Availability and Licensing:

  • License: Released under the Apache 2.0 license.
  • Integration: Available in agent mode within Mistral Vibe (via /leanstall or vibe --agent lean).
  • API: Accessible through the labs-leanstral-2603 free/near-free API endpoint.
  • Weights: Model weights are available for local deployment.

Leanstral supports arbitrary Model Context Protocols (MCPs) through Vibe and is specifically optimized for the lean-lsp-mcp.

Performance Benchmarks on FLTEval

To evaluate the model in realistic proof engineering scenarios, Mistral AI introduced FLTEval, a new evaluation suite that benchmarks the completion of formal proofs and the definition of new mathematical concepts within pull requests to the FLT project.

Comparison with Open-Source Models

Leanstral-120B-A6B demonstrates a significant efficiency advantage over larger open-source models. While GLM5-744B-A40B and Kimi-K2.5-1T-32B capped their FLTEval scores at approximately 16.6 and 20.1 respectively, Leanstral outperformed both in a single pass.

Qwen3.5-397B-A17B required four passes to reach a score of 25.4, whereas Leanstral achieved a superior score of 26.3 with only two passes (pass@2) and reached 29.3 at the same cost level (pass@4).

Comparison with Claude Family

Leanstral provides a high-value alternative to the Claude suite, offering competitive performance at a lower cost:

Model Cost ($) Score
Haiku 184 23.0
Sonnet 549 23.7
Opus 1,650 39.6
Leanstral 18 21.9
Leanstral pass@2 36 26.3
Leanstral pass@4 72 29.3
Leanstral pass@8 145 31.0
Leanstral pass@16 290 31.9

At pass@2, Leanstral (score 26.3) beats Sonnet (score 23.7) by 2.6 points at a cost of $36 compared to Sonnet's $549. At pass@16, Leanstral reaches a score of 31.9, beating Sonnet by 8 points.

Practical Capabilities and Case Studies

Leanstral is capable of diagnosing complex compilation issues and translating formal definitions between proof assistants.

Debugging Lean 4 Releases

In a real-world scenario involving a script that stopped compiling in Lean 4.29.0-rc6, Leanstral diagnosed a failure in the rw (rewrite) tactic due to a type alias created with def. The model recreated the failing environment with test code and correctly identified that def creates a rigid definition that blocks the rw tactic. Leanstral proposed and implemented a fix by replacing def with abbrev, which creates a transparent alias that allows the rw tactic to match the pattern successfully.

Program Reasoning and Translation

Leanstral successfully converted definitions from Rocq to Lean, including the implementation of custom notation. It can translate Rocq statements and subsequently prove properties about those programs in Lean, such as proving that a command adding 2 to a variable X correctly updates the state to n+2.

Sources

Related

  • Dispatch
    Leanstral 1.5 Release Notes: Advancing Formal Verification with Lean 4

    Mistral AI has released Leanstral 1.5, an Apache-2.0 licensed model that achieves state-of-the-art results in formal mathematics and discovers real-world bugs through automated code verification.

  • Dispatch
    Leanstral 1.5 release notes / what's new

    Mistral AI has released Leanstral 1.5, a specialized model for Lean 4 formal proof engineering, automated theorem proving, and autoformalization.

  • Dispatch
    Leanstral 1.5 Release: State‑of‑the‑art Formal Verification Model

    Mistral AI released Leanstral 1.5, a 119 B‑parameter, Apache‑2.0 model that saturates miniF2F, solves 587 of 672 PutnamBench problems, and discovers previously unknown bugs in open‑source code.

  • Dispatch
    Mistral AI Devstral Release

    Mistral AI has released Devstral, an agentic LLM for software engineering that outperforms open-source models on the SWE-Bench Verified benchmark and is available under the Apache 2.0 license.

  • Dispatch
    Mistral AI Devstral 2 and Mistral Vibe CLI Release

    Mistral AI has released Devstral 2 (123B) and Devstral Small 2 (24B), open-weight coding models that achieve state-of-the-art performance on SWE-bench Verified with high cost-efficiency, alongside the open-source Mistral Vibe CLI for autonomous code automation.