Codestral Mamba Release Notes

Mistral AI has introduced Codestral Mamba, a specialized model for code and reasoning designed to overcome the scaling limitations of traditional Transformer architectures. By utilizing the Mamba architecture, the model achieves linear time inference and theoretical support for infinite sequence lengths, making it highly efficient for long-context code productivity tasks.

Mamba Architecture and Performance Advantages

Codestral Mamba leverages the Mamba architecture to provide significant efficiency gains over standard Transformer-based models. The primary technical advantages include:

  • Linear Time Inference: Unlike Transformers, Mamba models allow for faster responses regardless of the input length.
  • Infinite Sequence Modeling: The architecture provides the theoretical ability to model sequences of infinite length.
  • SOTA Performance: Despite the architectural shift, Codestral Mamba is trained to perform on par with state-of-the-art (SOTA) transformer-based models in coding and reasoning tasks.

Technical Specifications and Capabilities

Codestral Mamba is an instructed model optimized for local deployment as a code assistant. Key specifications include:

  • Parameter Count: 7,285,403,648 parameters.
  • Context Window: The model has been tested for in-context retrieval capabilities up to 256k tokens.
  • Development: The model was developed with contributions from Albert Gu and Tri Dao.

Deployment and Availability

Codestral Mamba is available for free use, modification, and distribution under the Apache 2.0 license. Users can access and deploy the model through several channels:

  • Weights: Raw weights are available for download via HuggingFace.
  • SDKs: Deployment is supported via the mistral-inference SDK (v1.2.0) and NVIDIA's TensorRT-LLM.
  • API Access: The model is available for testing on la Plateforme under the identifier codestral-mamba-2407.
  • Local Inference: Support for llama.cpp is expected in the future.

Sources

Related

  • Dispatch
    Mistral 7B Release Notes

    Mistral AI has released Mistral 7B, a 7.3B parameter language model that outperforms Llama 2 13B across all benchmarks and is available under the Apache 2.0 license.

  • Dispatch
    Mathstral 7B Release

    Mistral AI has released Mathstral, a 7B parameter model specializing in STEM subjects and advanced mathematical reasoning, developed in collaboration with Project Numina.

  • Dispatch
    Codestral Release Notes

    Mistral AI has released Codestral, a 22B open-weight generative AI model specifically designed for code generation across 80+ programming languages.

  • Dispatch
    Mistral Large 2 release notes / what's new

    Mistral AI has released Mistral Large 2, a 123B parameter model designed for high throughput single-node inference with competitive performance in coding, reasoning, and multilingual capabilities.

  • Project
    state-spaces/mamba

    Mamba is a linear-time sequence modeling architecture based on selective state space models (SSMs) that provides a high-performance, subquadratic alternative to Transformers.