zh-plus/openlrc

What it solves

Open-Lyrics provides a streamlined way to convert audio or video files into synchronized subtitles (specifically .lrc files). It solves the problem of inaccurate or robotic translations in automated transcription by combining high-speed speech-to-text with the contextual polishing capabilities of Large Language Models (LLMs).

How it works

The library follows a multi-stage pipeline:

1. Audio Preprocessing: It uses loudness normalization and optional noise suppression (via DPDFNet) to reduce hallucinations during transcription.
2. Transcription: It employs faster-whisper to convert audio into text with timestamps.
3. Translation & Polishing: The transcribed text is sent to an LLM (such as GPT-4, Claude, or Gemini) to be translated and polished. It uses context-aware prompts and can utilize a custom glossary to ensure domain-specific terms are translated correctly.
4. Output: The final result is saved as a synchronized subtitle file.

Who it’s for

• Developers building subtitle generation tools.
• Content creators who need high-quality, translated lyrics or subtitles for audio/video content.
• Users who want to automate the transcription and translation of podcasts, songs, or videos using a mix of local ASR and cloud-based LLMs.

Highlights

• Mixed-Model Support: A "lean translation mode" allows users to use a cheap model for initial translation and a larger model for correction to save tokens.
• Contextual Accuracy: Maintains context between translation segments to avoid disjointed subtitles.
• Flexible Integration: Supports multiple LLM providers (OpenAI, Anthropic, Google, OpenRouter) and local OpenAI-compatible endpoints.
• Custom Glossaries: Supports JSON/YAML glossaries to ensure consistent translation of specialized terminology.