meizhong986/WhisperJAV

What it solves

WhisperJAV is a subtitle generator specifically designed for Japanese Adult Videos (JAV). It addresses common failures of general-purpose speech recognition models (like Whisper) when dealing with the unique acoustic profiles of this genre, such as low signal-to-noise ratios, frequent non-verbal vocalizations (moans, gasps), extreme volume swings, and the tendency for models to hallucinate text during long periods of ambiguous audio or silence.

How it works

Instead of relying on a single model, the project implements a multi-stage pipeline to harden the transcription process:

1. Scene-based Segmentation: Splits audio at natural acoustic boundaries to ensure the model processes coherent environments.
2. VAD (Voice Activity Detection) Clamping: Detects actual speech and feeds only those segments to the model to prevent hallucinations on non-speech audio.
3. Speech Enhancement (Optional): Can clean audio specifically for the VAD to improve detection while keeping the original audio for the actual transcription to avoid stripping necessary high-frequency detail.
4. ASR Models: Supports various engines including Faster-Whisper, OpenAI Whisper, Qwen3-ASR, and anime-whisper.
5. Post-processing: Applies Japanese-specific cleanup, including sentence regrouping based on ending particles, removal of sound-only lines (e.g., pure moans), and timing repairs.
6. Ensemble Mode: Allows running two different pipelines (different models and VADs) and merging the results to fill gaps that a single model might miss.

Who it’s for

Users who need high-quality Japanese subtitles for adult content and want a free, local solution that ensures privacy by avoiding cloud uploads.

Highlights

• Local Execution: Runs on the user's own machine with a GUI or CLI.
• ChronosJAV: A decoupled design that separates text generation from timing, allowing the use of modern recognizers like Qwen3-ASR.
• Two-Pass Ensemble: Merges results from multiple models to increase coverage.
• Japanese-Aware Filtering: Specifically tuned to handle the nuances of Japanese dialogue and non-verbal sounds common in the target domain.
• Flexible Pipeline: Mix-and-match options for scene detection, speech enhancement, and VAD providers.