kizuna-ai-lab/sokuji

What it solves

Sokuji provides real-time, two-way speech translation for bilingual meetings, allowing people speaking different languages to communicate seamlessly across various platforms like Zoom, Teams, and Discord. It eliminates language barriers by providing live subtitles and translated audio output.

How it works

The app captures audio from the microphone and system audio, then processes it through a translation pipeline (Automatic Speech Recognition $\rightarrow$ Translation $\rightarrow$ Text-to-Speech). Users can choose between two processing modes:

• Cloud Mode: Connects directly to providers like OpenAI, Google Gemini, and Soniox for high-performance translation.
• Local Inference: Runs everything on-device using WASM and WebGPU (via sherpa-onnx and Transformers.js), requiring no API keys, no internet connection, and no dedicated GPU.

Who it’s for

It is designed for professionals and individuals participating in multilingual meetings, gamers, or anyone needing real-time translation of their own voice or the audio from other participants in any desktop application or web-based meeting platform.

Highlights

• Two-Way Translation: Auto-detects speaker languages and translates in both directions simultaneously.
• Hybrid Processing: Supports both cloud-based APIs and fully offline local inference for maximum privacy.
• Cross-Platform: Available as a desktop app (Windows, macOS, Linux) and a browser extension (Chrome, Edge).
• Extensive Model Support: Access to dozens of ASR, translation, and TTS models locally, including Whisper and Qwen.
• Audio Tooling: Includes AI noise suppression, echo cancellation, and a virtual microphone to route translated audio into other apps.