inclusionAI/Ming-3.0-tts

What it is

Ming-3.0-tts is a text-to-speech system that turns written text into spoken audio. Its headline feature is control: you can steer who speaks (clone a voice, design one from a description, or pick a built-in voice), how it's said (emotion, speed, pitch, volume, or a natural-language instruction like "speak in a warm, intimate storytelling voice"), and what goes beyond words (inline tags for laughter, sighs, coughs, etc.). It also handles multiple languages and dialects, and — in the larger model — can generate multi-speaker dialogues from a script.

The two models

Model	Architecture	Size	Notes
Ming-3.0-tts-0.5b	Dense (every parameter used for every output)	0.5 billion	Simpler to run; the main quick-start path
Ming-3.0-tts-tiny	MoE (Mixture of Experts — only a subset of parameters activate per token)	7.9B total, 1.3B active	Built on the Ling V3 backbone; adds the podcast multi-speaker task

Both share the same generation framework: text plus task-specific control tokens feed a language model, which drives a DiT (diffusion-transformer) patch head and an audio tokenizer to produce speech.

Key capabilities

• Voice creation — three routes: clone a reference voice from a short audio clip, design a voice from a text description (e.g. "mature, steady male documentary narrator"), or pick from 100+ Chinese and 5 English built-in voices.
• Free-form control — describe the desired delivery in plain language; no rigid parameter list.
• Paralinguistic control — 14 inline tags ([Laughter], [Sigh], [Cough], [Surprise-ah], etc.) that can be placed anywhere in the sentence, in Chinese or English.
• Languages & dialects — Chinese, English, German, Spanish, French, Korean, Japanese, plus Sichuanese and Cantonese.
• Emotion & attributes — explicit emotion labels (happy, sad, angry, fearful, disgusted, surprised, neutral) and speed/pitch/volume settings.
• Multi-speaker dialogue — podcast task (tiny only): write a script with speaker_1: / speaker_2: lines and provide one reference audio per speaker.

Important constraint: each capability is a separate task mode. You can't freely mix, e.g., emotion + dialect + voice design in one call — the system rejects controls from other tasks.

Independent leaderboard results

On the third-party Artificial Analysis TTS leaderboards (October 2026):

• Controlled Voice — 1st among open-weights models (13th overall); every model clones the same 8 voices.
• Pronunciation Robustness — 1st overall (93.2% correctly pronounced spans); also 1st on expanding shorthand (96.5%) and preserving exact sequences (90.6%).
• Provider Voice — 3rd among open-weights models (17th overall).

Ranks shift as new models are added.

Getting started

Install via pip install -r requirements.txt (0.5b) or a Docker image; tiny needs a separate pinned environment (Python 3.10.15, PyTorch 2.7.1+cu118). Download weights from Hugging Face or ModelScope. Text normalization uses FST grammars requiring pynini (optional; can be skipped).

A minimal 0.5b call:

from ming_tts import MingTTS
model = MingTTS("inclusionAI/Ming-3.0-tts-0.5b")
model.synthesize(
    task="TTS",
    language_id="Chinese",
    text="傍晚的风从山口吹过来，远处的云慢慢染成了金色。",
    prompt_wav_path="data/voice_presets/voice_audio/lingguang.wav",
    prompt_text="诶，你今天有空吗？我们能不能聊聊这件事啊？",
    use_spk_emb=True,
    output_path="output/tts.wav",
)

Tiny uses a similar API (MingTinyTTS.from_pretrained(...).generate(...)) with lower-case task names, plus one-click shell scripts in examples/one_click/.

Task modes (the "cookbook")

The README's cookbook organizes tasks by the production line: who speaks (cloning, voice design, built-in IP voices), how it's said (freeform, emotion, speed/pitch/volume), beyond the words (paralinguistic tags), and where it's heard (multilingual, dialects). Each example is a single task; mixing is rejected. Full value lists are in cookbooks/instruction_en.md / cookbooks/instructions.md, with an end-to-end notebook.

Performance highlights (open benchmarks)

• Paralinguistic: SOTA on all six NVBench single-tag metrics across the two models; tiny is SOTA on all three English multi-tag metrics.
• Dialect: 0.5b SOTA on Cantonese Base CER/ACC; tiny SOTA on Sichuanese Hard CER and Easy ACC.
• Multilingual: tiny SOTA on German and Spanish WER, 2nd on French WER, tied 2nd on Japanese CER.
• Emotion: tiny ties SOTA on CV3 text-related Happy, SOTA on both CV3N Sad subsets; 0.5b SOTA on CV3 text-related Angry.
• Attribute control: 0.5b leads on average accuracy and pitch; tiny leads on WER and speaker similarity.
• Podcast (tiny): CER 2.03%, cpSIM 0.561, UTMOS 2.12 — ranks 1st, 2nd, 2nd.
• Voice Design (tiny): 2nd on average accuracy (75.37%), APS (85.10%), DSD (79.90%).
• Chinese/English: 0.5b Chinese WER ranks 3rd; tiny SOTA on Chinese and English speaker similarity.

Ranks are computed only among the listed open-source models; Seed-TTS and NVbench-GT are reference rows, not ranked. Where a historical Ming-omni-tts result beats a Ming-3.0-tts value, the README notes it and claims no SOTA.

Roadmap status

Released: both models' weights and inference code, tiny benchmarks, and vLLM-Omni accelerated inference. Not yet released: a technical report.

Notes

• The README lists a Limitations section and a Usage Disclaimer in its table of contents, but the provided excerpt doesn't include their content — so I can't summarize them.
• The README also links a demo video and a blog with audio examples for side-by-side listening.