ddxfish/sapphire

What it solves

Sapphire is a self-hosted agentic framework designed to create highly customizable AI personas with long-term memory, voice interaction, and the ability to interact with the physical and digital world. It aims to move beyond simple chatbots by providing a system where AI agents can autonomously manage goals, execute shell commands, and integrate with smart home devices.

How it works

The system acts as a wrapper for Large Language Models (LLMs), supporting both local (via Ollama/LM Studio) and cloud providers. It uses a layered memory engine (Mind Palace) with semantic vector search and a "librarian" that organizes memories nightly. For interaction, it combines wake-word detection, speech-to-text (STT), and text-to-speech (TTS) for hands-free conversation. It is highly extensible through a plugin system that allows the AI to add its own tools, hooks, and UI components.

Who it’s for

It is designed for developers and AI enthusiasts who want a private, self-hosted AI companion or a smart home controller that can autonomously perform tasks and execute code on their own systems.

Highlights

• Autonomous Agency: Capable of executing shell commands, managing hierarchical goals, and spawning background AI workers (agents).
• Deep Memory: Features semantic vector search across 100K+ entries and a nightly organization process to maintain a consistent persona.
• Voice Integration: Full voice stack including wake words, barge-in conversation mode, and integration with Twilio for real phone calls.
• Extensible Ecosystem: A plugin store for community-made tools and the ability for the AI to write and install its own plugins at runtime.
• Crosspoint Integrations: Native support for Home Assistant, Discord, Telegram, Email, Google Calendar, and Bitcoin wallets.
• Multimodal Capabilities: Supports vision-capable LLMs for webcam image capture and 3D animated avatars for visual presence.