pipecat-ai/pipecat

Open Source framework for voice agents, multimodal apps, and realtime AI. Maintained by Daily and the community.

What it solves

Pipecat is a framework designed to simplify the creation of real-time, multimodal conversational AI agents. It removes the complexity of orchestrating audio, video, and AI services, allowing developers to build everything from simple voice assistants to complex multi-agent systems that can coordinate across different processes or machines.

How it works

Pipecat uses a modular, composable pipeline architecture. Each agent is represented as a pipeline that integrates pluggable services for speech-to-text (STT), large language models (LLMs), and text-to-speech (TTS), or direct speech-to-speech services. These pipelines can be composed to enable handoffs between specialist agents, parallel execution (fan-out), or sidecar worker patterns, all while maintaining ultra-low latency via transports like WebRTC and WebSockets.

Who it’s for

It is intended for developers building voice-first AI applications, such as AI companions, business support bots, interactive storytelling tools, and complex dialog systems.

Highlights

  • Multi-Agent Orchestration: Supports handoffs, parallel fan-out, and distributed deployments over a shared bus.
  • Voice-First Integration: Built-in support for a vast array of STT, LLM, and TTS providers.
  • Real-Time Performance: Optimized for ultra-low latency interaction.
  • Extensive Ecosystem: Includes client SDKs for multiple platforms (JS, React, Swift, Kotlin, C++), a CLI for project scaffolding, and a real-time debugger (Whisker).

Related

  • Project
    pipecat-ai/pipecat-examples

    A collection of intermediate and advanced example applications demonstrating how to build voice and multimodal AI agents using the Pipecat framework.

  • Project
    pipecat-ai/pipecat-client-web

    A web client SDK and React library for connecting to and interacting with voice and multimodal AI applications built with the Pipecat framework.

  • Project
    uezo/aiavatarkit

    A modular framework for building ultra-low latency conversational AI avatars that integrate VAD, STT, LLM, and TTS across multiple channels like web, phone, and hardware.

  • Project
    livekit/agents

    A framework for building real-time, programmable multi-modal AI agents that can see, hear, and speak, featuring integrated job scheduling and telephony support.

  • Project
    huggingface/speech-to-speech

    Speech‑to‑Speech is a Hugging Face open‑source pipeline that turns spoken input into spoken output. It chains VAD → STT → LLM → TTS, each component being swappable (Silero VAD, Parakeet/Faster‑Whisper/Whisper STT, any OpenAI‑compatible LLM or local Transformers/MLX model, Qwen3‑TTS or other TTS back‑ends). The whole system speaks the OpenAI Realtime protocol over WebSocket/WebRTC, so existing OpenAI Agents SDK code works. Install with a single `pip install speech-to-speech`, then run `speech-to-speech serve` (server) and `speech-to-speech talk` (client) or `speech-to-speech local` (both together). It supports local‑only operation, mixed local/hosted LLMs, Docker deployment, and optional extras for alternative TTS/STT models. Designed for voice‑assistant robots (e.g., Reachy Mini) and low‑latency spoken AI applications.