bigsk1/voice-chat-ai

🎙️ Speak with AI - Run locally using Ollama, OpenAI, Anthropic or xAI - Speech uses SparkTTS, OpenAI, ElevenLabs, Kokoro, Typecast or xAI

What it solves

Voice Chat AI provides a way to interact with AI characters through speech, enabling real-time, expressive conversations. It solves the problem of limiting AI interactions to text-based interfaces by integrating multiple speech-to-text (STT) and text-to-speech (TTS) providers to create a seamless voice-driven experience.

How it works

The system integrates various LLM providers (OpenAI, xAI, Anthropic, Ollama) with a range of TTS engines (Spark-TTS, OpenAI TTS, ElevenLabs, Kokoro TTS, Typecast) and STT options (OpenAI, Local Faster Whisper). It can be run as a Web UI or a terminal-based CLI. For high-performance, real-time interaction, it utilizes OpenAI's WebRTC Realtime API. It also includes sentiment analysis to adjust AI responses based on the user's mood.

Who it’s for

Users looking for immersive AI companionship, role-play, or interactive storytelling. It is also for developers who want a flexible framework to mix and match different AI chat and voice providers locally or via API.

Highlights

  • Multi-Provider Support: Mix and match LLMs from OpenAI, xAI, Anthropic, and Ollama.
  • Real-time Interaction: Supports OpenAI's WebRTC Realtime API for instant, interruptible conversations.
  • Llocal Voice Cloning: Local zero-shot voice cloning via Spark-TTS.
  • Interactive Content: Includes 15+ built-in game types and immersive story adventures.
  • Mood Analysis: Adjusts AI responses based on user sentiment analysis.
  • Visual Integration: Ability to analyze screen captures using models like LLaVA.
  • Flexible Deployment: Supports native installation for best audio performance or Docker for easier setup.

Related

  • Project
    datascale-ai/opentalking

    An open-source orchestration framework for real-time digital-human conversations, integrating LLMs, TTS, STT, and video rendering via WebRTC.

  • Project
    PunithVT/ai-avatar-system

    A production-ready platform for creating photorealistic AI avatars with real-time lip-sync, zero-shot voice cloning, and multi-LLM support.

  • Project
    jamiepine/voicebox

    An open-source, local-first AI voice studio for voice cloning, speech generation, and system-wide dictation, featuring an MCP server for AI agent integration.

  • Project
    tover0314-w/opentypeless

    An open-source AI voice input tool for macOS, Windows, and Linux that provides context-aware dictation, AI rewriting, and one-shot voice Q&A across any desktop application.

  • Project
    Hyk260/PureChat

    A cross-platform chat application and AI development framework that integrates multiple LLMs like OpenAI and DeepSeek with support for web search and streaming output.