Lynpoint/CyberVerse

Self hosted, real-time digital human agent platform. Build voice-first AI agents with WebRTC, persona memory, tools, RAG, and optional digital-human video.

What it solves

CyberVerse is designed to create real-time, voice-driven digital human agents. It allows users to turn a single photo into a living AI character that can see, hear, and speak in real-time, providing a more natural, human-like interaction experience than text-based chatbots.

How it works

The framework uses a modular, plugin-based architecture where components for the "brain" (LLM), voice (TTS), hearing (ASR), and face (avatar models) are replaceable. It employs a multi-agent system consisting of a foreground PersonaAgent for fluid conversation and background SubAgents for complex, asynchronous tasks like research or report generation. For the visual component, it integrates local models like FlashHead and LiveAct, or cloud APIs (Baidu Xiling, Xunfei), streaming the result via WebRTC for low-latency, full-duplex interaction.

Who it’s for

It is intended for developers and AI enthusiasts who want to build interactive digital humans, virtual assistants, or character-based AI agents with integrated voice and video capabilities.

Highlights

  • Single-Photo Animation: Create a real-time digital human from just one image.
  • Full-Duplex Interaction: Supports natural conversations where users can interrupt the AI in real-time.
  • Multi-Agent Architecture: Separates conversational fluidity from heavy background processing via PersonaAgent and SubAgents.
  • RAG Integration: Supports importing knowledge bases and biographical material to align AI responses with a specific character's persona.
  • Modular Stack: Easily switch between various LLM, TTS, and ASR providers, including support for 100+ providers via LiteLLM.
  • Flexible Deployment: Can be run as a pure voice agent or a full digital-human video agent depending on available GPU resources.

Related

  • Project
    TEN-framework/ten-framework

    An open-source framework for building real-time multimodal conversational AI agents with low-latency audio, text, and visual capabilities.

  • Project
    datascale-ai/opentalking

    An open-source orchestration framework for real-time digital-human conversations, integrating LLMs, TTS, STT, and video rendering via WebRTC.

  • Project
    livekit/agents

    A framework for building real-time, multi-modal voice agents that can see, hear, and understand, featuring integrated job scheduling and flexible model integrations.

  • Project
    GetStream/Vision-Agents

    A framework for building low-latency, multi-modal AI agents that can watch, listen, and understand video in real time using a combination of CV models and LLMs.

  • Project
    plurai-ai/intellagent

    A multi-agent framework that stress-tests conversational AI agents by simulating thousands of realistic edge-case interactions to uncover failure points and optimize performance.