bilibili/Index-Translate

A Multilingual, Multimodal Translation Model Family

What it solves

Index-Translate addresses the need for a unified, high-performance translation system that goes beyond simple text-to-text conversion. It provides a comprehensive solution for complex translation tasks, including multimodal inputs (images and audio), strict adherence to formatting and terminology constraints, syllable-controlled dubbing, and long-document consistency.

How it works

Built on the Qwen3.5 architecture, this model family uses specialized variants for different translation needs. It supports 150 languages through a text-based model, while its multimodal components use native vision encoders for image translation (avoiding external OCR) and end-to-end pipelines for speech-to-subtitles (S2TT) and voice-cloned speech-to-speech (S2ST) dubbing. The system allows users to inject hard constraints (like specific glossaries or JSON structure preservation) and soft constraints (like tone or domain-specific style) into the translation process.

Who it’s for

This project is for developers and researchers who need a robust, multilingual translation engine that can be self-hosted or accessed via API. It is particularly useful for those building applications that require high-fidelity translations, such as automated subtitle generation, localized content creation, or enterprise-grade document translation where terminology consistency is critical.

Highlights

  • Multimodal Capabilities: Native support for image-to-text translation and end-to-end audio-to-speech/subtitle translation.
  • Instruction Following: Advanced support for hard constraints (glossaries, formatting) and soft constraints (tone, style, domain).
  • Controlled Dubbing: Specialized models for aligning translations to specific syllable counts for timing accuracy.
  • Developer-Friendly: Offers a free, OpenAI-compatible public API and supports local deployment via vLLM and llama.cpp (GGUF).

Related

  • Dispatch
    Qwen-MT Turbo Release Notes

    Qwen has released Qwen-MT (qwen-mt-turbo), a lightweight MoE-based translation model supporting 92 languages with high customizability and low API costs.

  • Dispatch
    Qwen3.8-LiveTranslate model release

    Qwen announced Qwen3.8‑LiveTranslate, a real‑time simultaneous interpretation model that reduces average lagging to 2.3 seconds, adds speaker separation, bilingual screen output, and long‑context disambiguation across 60 input languages.

  • Dispatch
    Qwen3.5-LiveTranslate-Flash Release Notes

    Qwen3.5-LiveTranslate-Flash is a simultaneous interpretation model built on Qwen3.5-Omni that provides real-time, multimodal translation across 60 languages with ultra-low latency and voice cloning.

  • Project
    chenpipi0807/ComfyUI-Index-TTS

    A ComfyUI extension that implements IndexTTS for high-quality voice cloning and text-to-speech, featuring multi-lingual support, emotion control, and multi-character novel parsing.

  • Dispatch
    Qwen3-TTS Release Notes: Open-Source Voice Design, Cloning, and Generation

    Qwen has open-sourced the Qwen3-TTS family, featuring 0.6B and 1.7B models that enable high-fidelity voice cloning, natural language-based voice design, and ultra-low latency streaming speech generation across 10 languages.