rzru/nightingale

What it solves

Nightingale allows users to turn any song from their local or networked music library into a karaoke track. It automates the difficult process of removing vocals, transcribing lyrics, and synchronizing them to the music, eliminating the need for pre-made karaoke files.

How it works

The application uses a pipeline of neural networks to process audio files:

1. Stem Separation: It uses the UVR Karaoke model or Demucs to isolate lead vocals from the instrumental track.
2. Transcription & Alignment: It employs WhisperX or Parakeet v3 to transcribe lyrics with word-level timestamps. For CJK (Chinese, Japanese, Korean) languages, it provides per-character forced alignment and romanized readings.
3. Playback: A Tauri-based app (Rust + React) handles the playback of the instrumental track, displays synchronized lyrics, and provides real-time pitch scoring via microphone input.

Who it’s for

Music lovers and karaoke enthusiasts who want to sing along to any song in their existing music collection (including those hosted on Plex, Jellyfin, or Navidrome) without searching for external karaoke versions.

Highlights

• Automated Setup: Automatically downloads and bootstraps Python, ffmpeg, and ML models on first launch.
• Library Integration: Connects to local folders, Plex, Jellyfin, and Navidrome servers.
• Advanced Audio Controls: Supports key and tempo shifts with cached variants for quick retries.
• Visual Experience: Features 10 audio-reactive GPU shaders and Pixabay video backgrounds.
• Pitch Scoring: Real-time microphone detection with star ratings and per-song scoreboards.
• Multi-platform: Ships as a single binary for Windows, macOS, and Linux.