katipally/openlive

Opensource, on-device voice + vision layer for AI agents. Bring any model or coding agent; the whole speech loop (VAD, STT, TTS, barge-in) runs locally. An open alternative to ElevenLabs Agents, Gemini Live, and OpenAI Realtime.

What it solves

Interacting with AI agents via voice and vision is typically expensive and requires sending audio data to cloud-hosted platforms. OpenLive replaces these proprietary, metered pipelines with a local, open-source alternative that handles all voice activity detection, speech-to-text, and text-to-speech processing directly on your machine.

How it works

OpenLive acts as a local "ears, mouth, and eyes" layer for your AI. It uses a cascaded pipeline (speech-to-text, then model, then text-to-speech) to process interactions. You can connect it to your own API keys for various AI models or link it to existing coding agents (like Claude Code or Cursor) using the Agent Client Protocol. Because it runs locally, your audio never leaves your computer; only the final transcript and optional visual frames are sent to the AI provider.

Who it’s for

It is for developers and power users who want to voice-control their coding agents, automate machine tasks, or dictate text without paying per-minute audio fees or sacrificing privacy to cloud-based voice platforms.

Highlights

  • Three modes: Chat (voice calls), Flow (system-wide voice assistant that can act on your machine), and Dictate (voice-to-text for any input field).
  • Local processing: Voice activity detection, speech-to-text (Whisper/Nemotron/etc.), and text-to-speech (Kokoro/Piper/etc.) run entirely on your hardware.
  • Agent integration: Works with coding agents like Claude Code, GitHub Copilot, and Cursor, allowing you to voice-drive their operations and hear narrated progress.
  • Privacy-first: Audio never leaves your machine, and API keys are encrypted at rest.
  • Customizable: Supports voice cloning, multiple languages, and the ability to toggle specific tools and capabilities for your agents.

Related

  • Project
  • Project
  • Project
  • Project
  • Project