Robostral Navigate Release Notes

Mistral AI has announced Robostral Navigate, an 8B model designed for embodied navigation that allows robots to move through complex environments using only a single RGB camera and plain-language instructions. This model outperforms existing multi-sensor approaches—including those using LiDAR or depth sensors—while remaining more efficient.

Performance Benchmarks

Robostral Navigate achieves state-of-the-art performance on the Room-to-Room in Continuous Environments (R2R-CE) benchmark, which measures the ability to follow instructions in environments not seen during training.

  • Validation Unseen: 76.6% success rate.
  • Validation Seen: 79.4% success rate.

These results represent a 9.7 point lead over the best single-camera approach and a 4.5 point lead over the best system utilizing depth sensors or multiple cameras.

Navigation Methodology: Pointing and Displacements

Robostral Navigate utilizes a "pointing" mechanism to predict the next movement. The model infers image coordinates of the target location within the current camera view and determines the desired orientation upon arrival. This approach makes the policy robust to changes in world scale and camera intrinsics.

When the target location is outside the current field of view, the model falls back to local coordinate frame displacements, providing specific instructions such as moving a set number of meters forward or left and turning by a specific degree.

Model Architecture and Training

Robostral Navigate was built entirely in-house, initialized from a vision-language model specialized in grounding tasks like object localization, counting, and pointing.

Data Generation

The model was trained entirely in simulation using a custom data generation pipeline that produced approximately 2.4 million trajectories across 350,000 scenes.

Supervised Training Efficiency

To optimize training, Mistral AI implemented a prefix-caching algorithm using a tree-based attention-masking strategy. This method compresses an entire episode into a single sequence, allowing the model to train on all time steps in a single forward pass. This reduced the number of training tokens by 22x, shortening training timelines from months to days.

Online Reinforcement Learning

Following supervised training, the model's performance was further enhanced using CISPO, an online reinforcement learning algorithm. This allowed the model to learn from trial and error and recover from failures, which improved the success rate by 3.2% and mitigated distribution shift issues associated with vanilla behavior cloning.

Hardware Generalization and Applications

Robostral Navigate is designed to be hardware-agnostic, running on wheeled, legged, and flying robots regardless of their size. Because it relies solely on a single RGB camera, it is applicable across various settings including offices, residential buildings, commercial spaces, and outdoor environments. Mistral AI identifies these capabilities as critical for applications in manufacturing, delivery, logistics, and hospitality.

Sources

Related

  • Dispatch
    Mistral AI Robostral Navigate 8B Model Enables Single-Camera Embodied Navigation

    Mistral AI’s 8‑billion‑parameter Robostral Navigate model lets robots follow natural‑language instructions using only a single RGB camera, achieving 76.6% success on the unseen R2R‑CE benchmark and outperforming multi‑sensor baselines.

  • Dispatch
    Mistral AI Shieldstral 1.0 3B Release

    Mistral AI has released Shieldstral, a 3B open-weights multimodal safety classifier that uses a binary question-answering framework to enable policy-adaptive moderation without retraining.

  • Dispatch
    Mathstral 7B Release

    Mistral AI has released Mathstral, a 7B parameter model specializing in STEM subjects and advanced mathematical reasoning, developed in collaboration with Project Numina.

  • Dispatch
    Mistral 7B Release Notes

    Mistral AI has released Mistral 7B, a 7.3B parameter language model that outperforms Llama 2 13B across all benchmarks and is available under the Apache 2.0 license.

  • Dispatch
    Pixtral 12B Release Notes

    Mistral AI has released Pixtral 12B, a natively multimodal model that combines a new 400M parameter vision encoder with a 12B parameter decoder to deliver high-performance multimodal reasoning without compromising text capabilities.