Mistral AI Voxtral TTS Release

Mistral AI has launched Voxtral TTS, a 4B-parameter text-to-speech (TTS) model designed for high-fidelity, multilingual voice generation. The model is engineered to provide realistic, emotionally expressive speech with low latency, making it suitable for scalable enterprise voice agents and real-time interactions.

High-Naturalness Multilingual Voice Generation

Voxtral TTS supports state-of-the-art performance across nine languages: English, French, German, Spanish, Dutch, Portuguese, Italian, Hindi, and Arabic. The model focuses on authenticity by capturing cultural nuances, natural rhythms, and emotional dexterity, including the ability to interpret context (such as sarcasm or happiness) to avoid robotic delivery.

Voice Emulation and Adaptation

Voxtral TTS can adapt to a custom voice using a reference sample as short as 3 seconds. This adaptation captures not only the speaker's voice but also specific nuances such as:

  • Subtle accents
  • Inflections and intonations
  • Disfluencies present in the reference audio

Notably, the model demonstrates zero-shot cross-lingual voice adaptation. It can generate speech in one language while adopting the accent of a voice prompt provided in another language (e.g., generating English speech using a French voice prompt to produce natural French-accented English).

Performance Benchmarks and Human Evaluation

Because automated metrics like word-error-rate cannot fully capture the naturalness of speech, Mistral AI utilized comparative human evaluations by native speakers.

  • Naturalness: Human evaluations indicate that Voxtral TTS achieves superior naturalness compared to ElevenLabs Flash v2.5 while maintaining a similar Time-to-First-Audio (TTFA).
  • Quality: The model performs at parity with ElevenLabs v3, specifically in its support for emotion-steering for lifelike interactions.
  • Zero-Shot Custom Voice: In a side-by-side preference test across the nine supported languages, Voxtral TTS showed a wider quality gap over ElevenLabs v2.5 Flash in zero-shot multilingual custom voice settings.

Technical Architecture

Voxtral TTS is a transformer-based, autoregressive, flow-matching model built upon the Ministral 3B backbone. The architecture consists of three primary components:

  1. Transformer Decoder Backbone: 3.4B parameters.
  2. Flow-Matching Acoustic Transformer: 390M parameters.
  3. Neural Audio Codec: 300M parameters (symmetric encoder-decoder).

Processing Workflow: The model accepts a voice prompt (5 to 25 seconds) and a text prompt. The transformer backbone predicts a semantic token for each audio frame; the flow-matching transformer then performs 16 function evaluations (NFEs) to produce the acoustic latent. An in-house codec processes audio causally using a semantic VQ (8192 vocabulary) and an acoustic FSQ (36 dim and 21 levels) latent at a 12.5Hz frame rate.

Latency and Streaming Capabilities

Designed for real-time voice agent applications, Voxtral TTS is optimized for low-latency streaming:

  • Model Latency: 70ms for a typical input of 500 characters (10 seconds of audio).
  • Real-Time Factor (RTF): Approximately 9.7x.
  • Generation Length: The model natively generates up to two minutes of audio, with the API supporting longer generations via smart interleaving.

Enterprise Integration and Availability

Voxtral TTS is positioned as the output layer for enterprise voice pipelines, capable of integrating with Voxtral Transcribe for full speech-to-speech workflows or existing speech-to-text and LLM stacks.

Access and Pricing:

  • API Access: Available at $0.016 per 1k characters.
  • Testing: Available in Mistral Studio and Le Chat.
  • Open Weights: A version with several reference voices is available on Hugging Face under the CC BY NC 4.0 license.

Sources

Related

  • Dispatch
    Mistral AI Voxtral Release

    Mistral AI has released Voxtral, a family of open-weight speech understanding models in 24B and 3B sizes that provide state-of-the-art transcription and native semantic understanding under the Apache 2.0 license.

  • Dispatch
    Mistral AI Voxtral Transcribe 2 Release

    Mistral AI has released Voxtral Transcribe 2, featuring Voxtral Mini Transcribe V2 for high-efficiency batch processing and Voxtral Realtime for ultra-low latency live transcription.

  • Dispatch
    Codestral Release Notes

    Mistral AI has released Codestral, a 22B open-weight generative AI model specifically designed for code generation across 80+ programming languages.

  • Dispatch
    Mistral Small 3.1 Release Notes

    Mistral AI has released Mistral Small 3.1, an Apache 2.0 licensed model featuring multimodal understanding, a 128k token context window, and performance that surpasses GPT-4o Mini and Gemma 3.

  • Dispatch
    Pixtral Large Release Notes

    Mistral AI has released Pixtral Large, a 124B open-weights multimodal model that achieves state-of-the-art performance on MathVista, DocVQA, and VQAv2 while maintaining the text capabilities of Mistral Large 2.