death34018-hue/AionsHome

What it solves

AionsHome is a comprehensive, locally-hosted AI companion system that integrates multi-modal interaction (text, voice, vision) with home automation and personal monitoring. It solves the problem of fragmented AI tools by combining a chat interface, smart camera surveillance, device activity tracking, and personal memory management into a single, synchronized ecosystem accessible via PC and mobile browsers (PWA).

How it works

The system uses a Python FastAPI backend with a SQLite database for local data storage. It connects to various AI providers (SiliconFlow, Google Gemini, AiPro) for LLM capabilities.

Key architectural components include:

• Dual-Brain Monitoring: A "Sentinel" (lightweight model) periodically analyzes camera frames and device logs to decide if the "Core" (main LLM) should proactively contact the user.
• RAG Memory: A vector-based memory system using Gemini embeddings that automatically summarizes conversations and retrieves relevant context based on a hybrid scoring system (vector similarity, keywords, and importance).
• Streaming TTS: A server-side pipeline that splits AI responses into sentences and synthesizes them in parallel using CosyVoice2, pushing audio chunks via WebSockets/SSE for near-instant playback.
• Multi-modal Bridges: An Android app provides native bridges for recording audio and capturing camera frames (including ESP32-CAM support) to bypass browser limitations.

Who it’s for

Users who want a highly personalized, privacy-focused AI assistant that lives on their own hardware and can "see" and "hear" their environment through integrated cameras and microphones.

Highlights

• Proactive AI: The AI can initiate conversations based on camera events or device activity using the [CAM_CHECK] and [查看动态] commands.
• Hybrid Memory: Combines automatic periodic summarization with an "Instant Sentinel" that decides when to search the vector database for deep context.
• Multi-platform Sync: Real-time synchronization across PC and mobile via WebSockets, with PWA support for a native app experience.
• Integrated Tooling: Includes built-in modules for EPUB reading with AI annotations, fund monitoring via akshare, and a TRPG game engine.
• Voice Interaction: Features WebRTC VAD for stable voice activation and a half-duplex conversation mode.