Rvosy/Sakura

Sakura Desktop Pet – What It Is

Sakura is an open‑source desktop‑pet framework that turns a character (the “pet”) into an AI‑driven agent living on your screen. It watches the screen, reacts to system events, and can start conversations on its own. The pet’s personality, voice, and appearance come from a character pack (.char file) that bundles a portrait, a “role card”, and optional GPT‑SoVITS voice weights.

Core Idea

Traditional chat‑bots wait for you to type. Sakura flips that: the agent proactively decides when to speak, using visual context (screenshots) and a memory of past interactions. Think of it as a little companion that can remind you of tasks, look up information, or just chat when it senses you might need company.

Main Features (as described in the README)

Category	Highlights
Character & Appearance	• One‑click import of .char packs (portrait, role card, voice). • Built‑in “Character Studio” (a Tauri window) to create/edit/export packs. • UI supports solid colour, blur, acrylic, macOS native glass; bubble size and position are configurable.
Active Perception	• Periodic screen‑capture → visual summary added to LLM context. • On‑demand region capture from the input bar (supports multi‑monitor, high‑DPI). • Self‑initiated speaking based on a periodic “should I talk?” evaluation.
Conversation Experience	• Dual‑language replies: Japanese original + Chinese subtitles, plus tone tags and drawing commands.
• Quick local “pre‑reply” while the full LLM response is being generated (optional local model boost and TTS).
• Type‑writer effect and scrollable bubbles that keep the pet’s layout stable.
• Fancy input‑box with blur and status indicators.
Tool & Memory Capabilities	• Built‑in tools: screenshot, open URL, to‑do/reminder creation, note read/write.
• Hierarchical long‑term memory that stores persona info, experiences, tasks, and habits; can be auto‑organized or manually edited.
• Tool calls are validated and executed with the same permissions as the main app; untrusted plugins are discouraged.
• Multiple API provider profiles (different keys/models for chat, vision, memory‑organizing).
Voice (TTS)	• Adapter for GPT‑SoVITS; users supply the service, model files, and voice weights.
• Supports self‑hosted GPT‑SoVITS instances, making it usable on macOS/AMD hardware.
Plugins & Extensions	• MCP server interface – any MCP‑compatible server can be hooked (built‑in web‑search MCP).
• Five default local plugins; optional Playwright‑based browser automation plugin (installable separately).
• Mobile web client plugin – lets a phone send text/images and share the same character, history, and memory with the desktop.
Debug & History	• GUI log viewer showing the agent’s decision process and tool calls.
• searchable chat history.
• Auto‑updater (Tauri Updater) for Windows and signed macOS builds.

How to Get Started

1. Download a release – the README warns not to use the source zip; grab the platform‑specific installer or portable ZIP from the Releases page.
2. Run the installer (Windows) or unzip and launch sakura.exe (portable). macOS and Linux users should follow the detailed setup guide in docs/userdocs/SETUP.md.
3. First‑run wizard – pick or import a .char pack, then enter your LLM API details (base URL, key, model name). Existing configs are reused on later launches.
4. Interact – the pet will appear in a corner, start listening, and may speak on its own. You can also type or send screenshots manually.

Who Might Use It

• Developers or hobbyists who want a playful, visual front‑end for an LLM‑powered agent.
• Content creators looking for a character that can autonomously fetch info, set reminders, or generate on‑screen captions.
• Anyone who enjoys desktop companions and wants to experiment with AI‑driven “agents that see the screen”.

License & Community

The code is released under the MIT License, allowing free use, modification, and redistribution. The project acknowledges inspiration from the Shinsekai desktop‑pet ecosystem and includes an optional Playwright‑browser plugin derived from another MIT‑licensed repo.

---

All details above are taken directly from the repository’s README; no additional features have been inferred.