bigsk1/voice-chat-ai

What it solves

Voice Chat AI provides a way to have natural, spoken conversations with AI characters. It eliminates the friction of typing by integrating speech-to-text (STT), large language models (LLMs), and text-to-speech (TTS) into a single interface, allowing users to interact with diverse personalities or play interactive games and stories.

How it works

The system acts as an orchestrator between several AI services. It captures user audio, transcribes it using OpenAI or local Faster Whisper, processes the text through a chosen LLM (such as OpenAI, xAI, Anthropic, or Ollama), and then synthesizes a voice response using a TTS provider (such as ElevenLabs, Spark-TTS, Kokoro, or OpenAI). It also supports OpenAI's WebRTC Realtime API for instant, interruptible conversations and includes sentiment analysis to adjust AI responses based on the user's mood.

Who it’s for

It is designed for users who want an immersive AI companion experience, role-players, and developers who want to experiment with mixing local and cloud-based AI models for voice interaction.

Highlights

• Multi-Provider Support: Mix and match LLMs (OpenAI, xAI, Anthropic, Ollama) and TTS engines (ElevenLabs, Kokoro, Spark-TTS, etc.).
• Real-time Interaction: Supports OpenAI's Realtime API for low-latency, interruptible speech.
• Local Execution: Option to run everything locally using Ollama and local TTS models to ensure privacy and reduce cost.
• Character-Driven: Includes a large selection of built-in characters and specialized modes for interactive games and storytelling.
• Visual Awareness: Can analyze the user's screen via specific voice commands using LLaVA or OpenAI.
• Flexible Deployment: Available via Web UI, Terminal, or Docker.