Mathstral 7B Release

Mistral AI has released Mathstral, a 7B parameter model designed to solve advanced mathematical problems requiring complex, multi-step logical reasoning. Developed in collaboration with Project Numina, Mathstral is intended to support the science community and academic projects by providing state-of-the-art reasoning capabilities within its size category.

Technical Foundation and Specialization

Mathstral is built upon the Mistral 7B architecture and is specialized for STEM (Science, Technology, Engineering, and Mathematics) subjects. It is released as an instructed model, meaning it is pre-configured to follow specific instructions and can be further adapted via fine-tuning.

Performance Benchmarks

Mathstral 7B achieves state-of-the-art reasoning capacities for its size category across industry-standard benchmarks. Key performance metrics include:

  • MATH Benchmark: 56.6%
  • MMLU Benchmark: 63.47%

Impact of Inference-Time Computation

The model's performance improves significantly when additional computation is applied during the inference phase. Specifically, Mathstral 7B's MATH score increases to:

  • 68.37% when using majority voting.
  • 74.59% when using a strong reward model among 64 candidates.

Availability and Implementation

Mathstral is available for the science community with weights hosted on HuggingFace. Users can implement the model using the following tools:

  • Inference: The mistral-inference tool (v1.2.0).
  • Adaptation: The mistral-finetune tool for custom fine-tuning.

Development Philosophy

Mathstral exemplifies Mistral AI's philosophy of optimizing the tradeoff between performance and speed by building models for specific purposes. This approach is further supported by the fine-tuning capabilities available on la Plateforme.

Sources

Related

  • Dispatch
    Mistral 7B Release Notes

    Mistral AI has released Mistral 7B, a 7.3B parameter language model that outperforms Llama 2 13B across all benchmarks and is available under the Apache 2.0 license.

  • Dispatch
    Mixtral 8x7B Release Notes

    Mistral AI has released Mixtral 8x7B, a sparse mixture-of-experts model that outperforms Llama 2 70B on most benchmarks while delivering 6x faster inference.

  • Dispatch
    Mistral AI Magistral Release Notes

    Mistral AI has released Magistral, its first reasoning model available in open-weight (Small) and enterprise (Medium) versions, featuring transparent, multilingual chain-of-thought reasoning.

  • Dispatch
    Mixtral 8x22B Release Notes

    Mistral AI has released Mixtral 8x22B, a sparse Mixture-of-Experts model with 141B total parameters and 39B active parameters, providing high performance and cost efficiency under an Apache 2.0 license.

  • Dispatch
    Mistral Large 2 release notes / what's new

    Mistral AI has released Mistral Large 2, a 123B parameter model designed for high throughput single-node inference with competitive performance in coding, reasoning, and multilingual capabilities.