OpenMOSS/MOSS-Transcribe-Diarize

What it solves

It solves the problem of creating structured, speaker-aware transcripts from long-form audio recordings. Instead of using separate systems for automatic speech recognition (ASR) and speaker diarization (identifying who spoke when), it performs both tasks jointly in a single pass to produce time-aligned text with precise timestamps and consistent speaker labels.

How it works

The model uses an end-to-end architecture combining a Whisper-Medium audio encoder with a Qwen3-0.6B style causal decoder. It processes raw audio in 30-second chunks, using a temporal merge and MLP adaptor to bridge audio features into the text backbone. The output is a compact stream of text formatted as [start_time][Sxx]transcribed speech[end_time], where Sxx represents the speaker ID.

Who it’s for

It is designed for users needing high-reliability transcription for meetings, phone calls, podcasts, interviews, lectures, and video content, supporting over 50 languages.

Highlights

• End-to-End Processing: Combines transcription and diarization into one step for better efficiency and alignment.
• Multi-Speaker Support: Automatically assigns consistent speaker labels (e.g., [S01], [S02]) to different voices.
• Acoustic Awareness: Capable of emitting optional acoustic event annotations to provide context beyond speech.
• Deployment Flexibility: Supports serving via SGLang Omni and vLLM with OpenAI-compatible endpoints.
• Integrated Tooling: Includes a subtitle web app for reviewing, exporting (SRT/ASS/JSON), and burning subtitles into video files.