volcengine/rtc-aigc-demo

What it solves

This project provides a reference implementation for building interactive, voice-based AI agents. It solves the complexity of integrating multiple separate components—Real-Time Communication (RTC), Automatic Speech Recognition (ASR), Large Language Models (LLMs), and Text-to-Speech (TTS)—into a single, low-latency end-to-end pipeline for natural voice conversations.

How it works

The system uses the Volcengine AIGC-RTC Server to orchestrate the AI pipeline. It integrates a frontend (React) and a backend (Koa) to handle session management and configuration. The cloud service manages the heavy lifting: edge user access, resource scheduling, audio/video stream compression, and the conversion between speech and text. Developers configure specific AI behaviors and model parameters via JSON scene files and OpenAPI calls, allowing them to focus on tuning the LLM rather than managing the underlying infrastructure.

Who it’s for

Developers looking to quickly prototype or deploy AIGC applications that require real-time, multimodal interaction, such as voice assistants, digital humans, or multi-user AI chat rooms.

Highlights

• End-to-End Pipeline: Integrates ASR, LLM, and TTS into a streamlined voice-to-voice workflow.
• Multimodal Support: Leverages RTC technology to support audio, video, and digital human interactions.
• Session Isolation: The server assigns unique SessionIDs and RoomIDs per page session, allowing multiple simultaneous conversations.
• Flexible Configuration: Supports custom scenarios and third-party models (e.g., Coze Bots) via JSON configuration files.