pipecat-ai/pipecat-client-web

What it solves

It provides a standardized way for web applications to connect and communicate with voice and multimodal AI bots built using the Pipecat framework. It removes the complexity of managing device media streams, bot configurations, and session states on the client side.

How it works

The project is a monorepo containing a JavaScript/TypeScript SDK (client-js) and a set of React components and hooks (client-react). It acts as a bridge between the user's browser and a Pipecat server. To function, it requires a separate transport implementation (such as WebRTC via Daily) to handle the actual transmission of audio and video data.

It also implements the UI Worker Protocol (v1), which allows server-side AI workers to observe and drive the GUI of the web application through structured wire formats, including accessibility snapshot streaming and UI command handling.

Who it’s for

Developers building web-based interfaces for multimodal AI agents that require real-time audio, video, and interactive UI synchronization.

Highlights

• Dual-package support: Offers both a vanilla JS/TS SDK and dedicated React bindings.
• UI Worker Protocol: Enables server-side bots to interact with and control the client-side GUI.
• Transport Agnostic: Uses a base Transport class, allowing developers to plug in different media providers.
• Session Management: Built-in handling for bot connectivity, media stream management, and error states.