morettt/my-neuro

This project lets you create your own AI desktop companion with customizable characters and voice conversations that respond in just 1 second. Features include long-term memory, visual recognition, voice cloning and LLM training. Compatible with various Live2D customizations.

What it solves

my-neuro is a workbench for creating highly personalized AI companions that mimic human-like interaction. It aims to move beyond simple chat interfaces by integrating personality, voice, visual representation, and long-term memory to create a digital entity that can integrate into a daily life.

How it works

The project provides a suite of tools to customize an AI's identity. It supports both local LLM inference and fine-tuning (via the LLM-studio folder) as well as integration with closed-source APIs. The system integrates several specialized components:

  • Visuals & Voice: Uses Live2D models for visual representation and GPT-SoVITS for custom voice training and TTS.

  • ** Corollary Interaction:** Supports real-time interruption, low-latency local inference, and synchronized subtitles/audio.

  • Capabilities: Includes vision capabilities for image recognition, web search for real-time information, and MCP tool integration for desktop control (e.g., opening software).

  • Memory: Implements a long-term memory system to remember user preferences and personality traits.

Who it’s for

Users who want to build their own virtual AI partner or VTuber-style AI, as well as those who interested in local LLM deployment and fine-tuning for character-driven AI.

Highlights

  • Multimodal Interaction: Combines LLMs, TTS, Live2D, and vision for a cohesive AI persona.
  • Low Latency: Optimized for local inference to keep conversation delays under one second.
  • L2D Integration: Support for replacing and controlling Live2D models based on dialogue content.
  • Extended Utility: Includes features like AI singing, Bilibili streaming, and the ability to play games (e.g., Minecraft, Galgames) with the user.
  • Local-First Option: Full support for local LLM inference and fine-tuning to avoid third-party API dependency.