kizuna-ai-lab/sokuji

Sokuji – Real‑time two‑way speech translation

What it is – Sokuji is a cross‑platform desktop app and browser extension that lets participants in a bilingual meeting speak their own language and hear (or read) an instant translation in the other language. It can run entirely offline using on‑device models compiled to WebAssembly/WebGPU, or it can forward the audio to cloud providers such as OpenAI, Google Gemini, Palabra.ai, Soniox, etc.

Why it matters – Real‑time speech translation usually requires a cloud API and an internet connection, which raises latency, cost, and privacy concerns. Sokuji’s Local Inference mode removes those barriers: no API key, no GPU, no data leaves the device, yet it still supports dozens of languages through a large collection of ASR, translation, and TTS models.

---

Core capabilities

Category	Details
Audio pipeline	Speech‑to‑text → Machine translation → Text‑to‑speech, all in a single stream. Supports two‑way translation (auto‑detects which speaker is using which language).
Local models	44 ASR models (including Whisper, Cohere Transcribe, Voxtral), 75 translation models (Opus‑MT, Qwen, TranslateGemma), 137 TTS models (Piper, Coqui, VITS, etc.).
Cloud providers	OpenAI, Google Gemini, Palabra.ai, Kizuna AI, Doubao AST 2.0, Soniox, Zoom AI Services, generic OpenAI‑compatible endpoints.
Languages	99+ languages for speech recognition, 55+ for translation, 53 for TTS.
Platforms	Windows, macOS (Intel & Apple Silicon), Linux (deb packages), Chrome/Edge/Brave extensions.
Privacy	In Local Inference mode everything runs on‑device; cloud mode connects directly to the provider without any intermediate server.
Audio features	Virtual microphone output, real‑time noise suppression, echo cancellation, live subtitles, system‑audio capture.

---

How it works (high‑level flow)

1. Capture – microphone (your voice) and/or system audio (other participants).
2. Choose mode – Cloud (send audio to a provider) or Local (run ASR/translation/TTS in the browser/Electron via WASM + WebGPU).
3. Process – ASR produces text, the translation model converts it, the TTS model synthesises speech in the target language.
4. Output – synthesized audio is sent to a virtual mic so Zoom, Teams, Meet, Discord, etc., play the translated voice; subtitles can be displayed or shared on screen.

---

Getting started

Target	Installation steps
Desktop app	Download the appropriate installer from the Releases page (Windows .exe, macOS .pkg, Linux .deb). Run the installer and launch Sokuji.
Browser extension	Add from the Chrome Web Store or Microsoft Edge Add‑ons, or load the zipped extension in developer mode (see the README).
Build from source	```bash
git clone https://github.com/kizuna-ai-lab/sokuji.git
cd sokuji && npm install
npm run electron:dev # dev mode
npm run electron:build # production build

---
### Typical use case
1. Open Sokuji and set *Language A* (e.g., Japanese) and *Language B* (e.g., English).  
2. Start a meeting in Zoom, Teams, Google Meet, Discord, etc.  
3. Speak in your native language; Sokuji detects the language, translates, and streams the translated voice back into the meeting.  
4. The other participant hears you as if you spoke their language, and you hear their speech translated into yours – all in real time.

---
### License & community
- **License:** AGPL‑3.0 (source code must remain open when distributed).  
- **Contributing:** Guidelines are provided; the project welcomes pull requests and bug reports.  
- **Support:** Issues on GitHub, Discussions for ideas, and a sponsor (SignPath) that provides free code‑signing for Windows builds.

---
### TL;DR
Sokuji is a ready‑to‑use, privacy‑first solution for live bilingual meetings. It can run completely offline using a large suite of on‑device speech models, or it can hook into major cloud AI services. Install the desktop app or browser extension, pick your two languages, and let the tool handle real‑time translation, subtitles, and virtual‑mic routing across Zoom, Teams, Meet, Discord, and many other platforms.