Blue-B/WhisperSubTranslate

What it solves

WhisperSubTranslate is a local application that converts video audio into multilingual subtitles. It solves the problem of needing expensive or cloud-based services to generate and translate subtitles while ensuring that video data remains private on the user's machine.

How it works

The app uses whisper.cpp for local speech-to-text transcription (generating SRT files) and offers two paths for translation:

1. Offline Translation: Uses bundled Tencent Hy-MT2 models (1.8B or 7B) to translate text on-device.
2. Online Translation: Integrates with external APIs like DeepL, OpenAI, Gemini, and Claude, or any OpenAI-compatible provider (e.g., Ollama, vLLM).

It also includes specialized "Sync repair" models (large-v2 Sync and Sync Lite) to fix subtitle drift in videos where standard models fail to maintain timing.

Who it’s for

Content creators and users who need to generate subtitles for videos in 100+ recognized languages and translate them into 15 target languages, specifically those who prioritize privacy, offline capability, and local GPU acceleration (CUDA/Vulkan).

Highlights

• 100% Local Processing: Speech-to-text and optional translation can be performed entirely offline without accounts or uploads.
• Flexible Translation Engines: Choice between bundled local Hy-MT2 models and various high-end cloud APIs.
• Automatic Setup: Handles model downloads automatically, requiring no manual Python installation for users.
• Sync Repair: Dedicated engines to correct timing drift in non-English videos.
• Hardware Acceleration: Supports NVIDIA CUDA and Vulkan-capable GPUs (AMD, Intel) for faster processing.