worm128/AI-YinMei

What it solves

AI-YinMei is a comprehensive platform designed to create AI-powered virtual streamers and interactive robots. It solves the difficulty of integrating multiple AI modalities—such as LLMs, text-to-speech (TTS), vision, and live-streaming tools—into a single, cohesive system for real-time interaction with audiences across various platforms.

How it works

The system acts as a central hub that aggregates inputs (like live-stream comments from Bilibili, TikTok, and others) and processes them through a pipeline of AI models. It uses a multi-head attention mechanism for intent and sentiment analysis, routes requests to compatible LLMs (OpenAI-standard), and generates responses using high-performance streaming TTS (including modified CosyVoice2). It can trigger external actions via MCP tools, control Live2D avatars, and manage long-term memory using a combination of MongoDB and Neo4j for knowledge graphs.

Who it’s for

• Virtual Streamers (VTubers): Those wanting to automate interactions, perform AI-driven singing, and manage a live audience.
• AI Enthusiasts: Users looking for a "out-of-the-box" AI desktop pet or QQ robot with multimodal capabilities.
• Content Creators: People who want to integrate AI-generated art, music, and video into their live broadcasts.

Highlights

• Multimodal Integration: Combines LLMs, vision, voice recognition, and image generation (Stable Diffusion) in one platform.
• Aggregated Bullet Chat: Supports multiple streaming platforms (Bilibili, Douyin, Kuaishou, etc.) and QQ robots.
• High-Performance Audio: Features streaming TTS with response times as low as 600ms and modified CosyVoice2 for better emotion.
• Advanced Memory & Logic: Implements RAG knowledge bases, long-term memory, and "diffuse thinking" via Neo4j graph databases.
• Extensive Toolset: Includes 27+ built-in tools for tasks like web searching, camera monitoring, and computer control.