fikrikarim/parlor
On-device, real-time multimodal AI. Have natural voice and vision conversations with an AI that runs entirely on your machine. Powered by Gemma 4 E2B and Kokoro.
What it solves
Parlor enables real-time, multimodal AI conversations (voice and vision) to run entirely on a local device. This eliminates the need for expensive server costs and cloud dependencies, making it possible to have a private, low-latency AI assistant that can see and hear the user.
How it works
The system uses a FastAPI server that communicates with a browser via WebSockets to stream audio and camera frames. It integrates three main components:
- Understanding: Gemma 4 E2B (via LiteRT-LM) processes speech and vision inputs.
- Speech: Kokoro TTS (using MLX on Mac or ONNX on Linux) converts text responses back into audio.
- Interaction: Silero VAD handles voice activity detection in the browser for hands-free use, and the system supports "barge-in," allowing users to interrupt the AI mid-sentence.
Who it’s for
It is designed for users with Apple Silicon Macs or Linux machines with supported GPUs who want a local multimodal AI, specifically those interested in language learning or private on-device assistants.
Highlights
- Fully Local: Runs entirely on-device with approximately 3 GB of RAM required.
- Multimodal: Simultaneously processes voice and visual input from the camera.
- Low Latency: Features sentence-level TTS streaming to start playback before the full response is generated.
- Hands-Free: Integrated Voice Activity Detection (VAD) removes the need for push-to-talk.