Leanstral 1.5 Release Notes: Advancing Formal Verification with Lean 4

Mistral AI has released Leanstral 1.5, a free Apache-2.0 licensed model designed for proof engineering in Lean 4. The model features 119B total parameters with only 6B active parameters, significantly improving the accessibility and power of formal verification for both mathematical theorems and software code.

State-of-the-Art Performance in Formal Mathematics

Leanstral 1.5 achieves state-of-the-art results across several rigorous mathematical benchmarks, demonstrating a high capacity for complex reasoning and long-horizon proof construction.

  • miniF2F: The model completely saturates this cross-system benchmark, reaching 100% on both validation and test sets.
  • FATE-H and FATE-X: Leanstral 1.5 sets a new state-of-the-art, solving 87% of graduate-level (FATE-H) and 34% of PhD-level (FATE-X) abstract algebra problems.
  • PutnamBench: The model solves 587 out of 672 problems from the Putnam Mathematical Competition. It outperforms Seed-Prover 1.5 (high setting) by 7 problems while operating at a fraction of the cost—approximately $4 per problem compared to an estimated $300+ for Seed-Prover.
  • FLTEval: Leanstral 1.5 improves pass@1 from 21.9 to 28.9 and pass@8 from 31.9 to 43.2, surpassing Opus 4.6.

Test-Time Scaling

Leanstral 1.5 exhibits strong test-time scaling, meaning its performance improves monotonically as the token budget per attempt increases. On PutnamBench, Pass@8 climbed from 44 problems solved at 50k tokens to 587 problems solved at 4M tokens. This allows the model to persist through millions of tokens of reasoning, editing, and revising to solve highly complex proofs.

Training Methodology: RL and Agentic Environments

Leanstral 1.5 was developed using a three-stage process: mid-training, supervised fine-tuning, and reinforcement learning (RL) with CISPO. The model's capabilities are derived from training in two distinct RL environments:

  1. Multiturn Environment: The model is tasked with proving or disproving a theorem. It submits a proof, receives feedback from the Lean compiler, and refines its approach iteratively until the proof compiles or the budget is exhausted.
  2. Code Agent Environment: The model operates as a developer within a raw filesystem. It can edit files, execute bash commands, and use the Lean language server to inspect goals and errors in real time. This environment enables the model to handle long-horizon tasks, such as building auxiliary lemmas and completing partial proofs in a repository.

Real-World Application: Code Verification and Bug Discovery

Beyond mathematics, Leanstral 1.5 is applied to software verification to identify hidden flaws in open-source repositories.

Proving Time Complexity

Leanstral 1.5 successfully proved the O(log n) time complexity guarantees for a real implementation of AVL trees. This required the model to use structural induction, handle monadic time tracking, and perform exhaustive case analysis for rebalancing paths over 2.7 million tokens and 22 context compactions.

Automated Bug Detection

Using a pipeline where Aeneas translates Rust code to Lean and Leanstral infers correctness properties, the model tested 57 repositories. It flagged 47 violated properties, 11 of which were genuine bugs, including five previously unreported on GitHub.

One notable discovery was an overflow bug in the datrs/varinteger library's sign function. On input Std.U64.MAX, the expression (value + 1) overflowed, leading to crashes in debug mode or silent corruption in release mode.

Community Insights and Counterpoints

While the release has been praised for its efficiency and the power of Lean 4, some community members have raised questions regarding the benchmarks and the bug-discovery examples:

"In what way would this boundary condition case be considered something that 'testing [...] would typically miss'? It's certainly something that bad tests would miss or not think about, but I find that (a) careful people and (b) ML coding systems are actually really good at 'oh, I should test the extreme values'"

Other critics noted that some of the compared models in the benchmarks are older generations, and some suggested that the bug found in the datrs/varinteger library may have been previously reported or was a trivial edge case that general-purpose LLMs could also identify.

Getting Started with Leanstral 1.5

Leanstral 1.5 is available under the Apache-2.0 license. Users can access the weights on Hugging Face or use the free API endpoint leanstral-1-5 via Mistral Vibe.

Installation Workflow:

  1. Install Mistral Vibe: uv tool install mistral-vibe $\rightarrow$ vibe --setup
  2. Install Leanstral 1.5: /leanstallexit
  3. Launch the agent: vibe --agent lean
  4. (Optional) Lean LSP MCP: Add lean-lsp-mcp to ~/.vibe/config.toml for enhanced language server support.

Sources

Related

  • Dispatch
    Leanstral 1.5 Release: State‑of‑the‑art Formal Verification Model

    Mistral AI released Leanstral 1.5, a 119 B‑parameter, Apache‑2.0 model that saturates miniF2F, solves 587 of 672 PutnamBench problems, and discovers previously unknown bugs in open‑source code.

  • Dispatch
    Leanstral 1.5 release notes / what's new

    Mistral AI has released Leanstral 1.5, a specialized model for Lean 4 formal proof engineering, automated theorem proving, and autoformalization.

  • Dispatch
    Mistral AI Leanstral Release

    Mistral AI has released Leanstral, an open-source code agent with 6B active parameters designed for Lean 4 to enable formally verified code and mathematical proofs.

  • Dispatch
    Mistral AI Devstral Release

    Mistral AI has released Devstral, an agentic LLM for software engineering that outperforms open-source models on the SWE-Bench Verified benchmark and is available under the Apache 2.0 license.

  • Dispatch
    Mistral Small 3 Release Notes

    Mistral AI has released Mistral Small 3, a latency-optimized 24B-parameter model under the Apache 2.0 license that rivals the performance of models three times its size.