pipecat-ai/pipecat-client-web

Real-Time Voice Inference Web SDK

What it solves

It provides a standardized way for web applications to connect and communicate with voice and multimodal AI bots built using the Pipecat framework. It removes the complexity of managing device media streams, bot configurations, and session states on the client side.

How it works

The project is a monorepo containing a JavaScript/TypeScript SDK (client-js) and a set of React components and hooks (client-react). It acts as a bridge between the user's browser and a Pipecat server. To function, it requires a separate transport implementation (such as WebRTC via Daily) to handle the actual transmission of audio and video data.

It also implements the UI Worker Protocol (v1), which allows server-side AI workers to observe and drive the GUI of the web application through structured wire formats, including accessibility snapshot streaming and UI command handling.

Who it’s for

Developers building web-based interfaces for multimodal AI agents that require real-time audio, video, and interactive UI synchronization.

Highlights

  • Dual-package support: Offers both a vanilla JS/TS SDK and dedicated React bindings.
  • UI Worker Protocol: Enables server-side bots to interact with and control the client-side GUI.
  • Transport Agnostic: Uses a base Transport class, allowing developers to plug in different media providers.
  • Session Management: Built-in handling for bot connectivity, media stream management, and error states.

Related

  • Project
    pipecat-ai/pipecat

    Pipecat is an open‑source Python framework for building real‑time voice‑first and multimodal AI agents. It offers plug‑and‑play support for dozens of speech‑to‑text, LLM, and text‑to‑speech services, composable pipelines, and built‑in multi‑agent orchestration. The ecosystem includes client SDKs (JS, React, Swift, Kotlin, C++, ESP32), a CLI for scaffolding projects, a UI component library, a debugger (Whisker), and many example bots. Ideal for developers and researchers who want to prototype or deploy voice‑centric conversational systems quickly.

  • Project
    pipecat-ai/voice-ui-kit

    A React-based UI kit providing components and templates for quickly building and deploying user interfaces for Pipecat AI voice applications.

  • Project
    pipecat-ai/pipecat-examples

    A collection of intermediate and advanced example applications for building voice and multimodal AI agents using the Pipecat framework.

  • Project
    livekit/python-sdks

    LiveKit’s Python SDK provides two packages (`livekit` for real‑time media participation and `livekit-api` for server‑side token and room management). It enables Python apps to join LiveKit video/audio rooms, publish/consume media, run RPC between participants, and integrate with LiveKit’s voice‑AI Agents (STT, LLM, TTS). Install via pip, generate a JWT token, connect to a room, and use the `MediaDevices` helper for microphone/speaker handling. The SDK is async, works with Python 3.9+, and is MIT‑licensed.

  • Project
    Mininglamp-OSS/octo-web

    A TypeScript and React-based web and desktop client for the OCTO messaging platform, designed specifically for human-AI agent collaboration.