Code-Amadeus/Amadeus

Real-time multimodal desktop agent evolving toward a persistent AI OS interface (0.17 α).

What it solves

Voice assistants, desktop characters, and execution agents usually live in separate windows: one chats, one performs, another runs in a terminal. This makes it hard to see what is happening during long tasks, which permission is needed, or whether work can recover after a failure. Amadeus connects these experiences into one loop so that speaking or typing, seeing an embodied character, delegating real work, and staying in control all happen in the same place.

How it works

Amadeus runs a local desktop app and a local Host. The user speaks or types to a Main Chat role, which delegates tasks to specialized Work Providers (such as Pi for daily tasks, Codex for complex coding, and a Browser for managed pages) rather than giving every tool directly to the chat model. Voice and character presentation are handled by dedicated components: ASR for wake and conversation, TTS for synthesis (including local GPT-SoVITS), and SpriteForge for animated character rendering with lip sync and expressions. A durable work control plane manages projects, drafts, permissions, artifacts, and recovery. The architecture deliberately separates authority domains: Main Chat, Work Providers, and AUIP application sessions each have distinct permissions, and a shared MCP/Skills registry is not directly accessible to Main Chat. All state, identity, and permissions are Host-verified facts, not model narration.

Who it's for

This is for developers and users comfortable building and running source code. It targets alpha stage (0.17.1 Alpha) and requires Python 3.12, Node.js, and a GPU for local voice models (or remote services). It supports Windows as the reference platform, macOS with Apple Silicon MPS, and Linux verified on real hardware. Users who want a privacy-conscious, local AI assistant with voice and character, and who can handle installation profiles, will benefit.

Highlights

  • Interruptible real-time voice conversation with barge-in across LLM, TTS, and playback
  • Animated character with lip sync, subtitles, and emotion driven by a shared playback timeline
  • Delegation to specialized Providers (Pi, Codex, Browser) with visible progress and permission requests
  • Durable work control: projects, drafts, artifacts, continue/retry, restart recovery, and structured diffs
  • AUIP cooperative application protocol for safe interaction with external applications
  • Optional local models: GPT-SoVITS for TTS, Qwen3-ASR for recognition, Silero VAD for wake word
  • Unified settings in Electron and clear separation of authority domains for security
  • Open-source under AGPL-3.0 with buildable, runnable source

Related

  • Project
    outsourc-e/hermes-workspace

    A comprehensive command center for AI agents that integrates chat, memory, skills, and a terminal into a single workspace for orchestrating multi-agent swarms.

  • Project
    bytedance/UI-TARS-desktop

    A multimodal AI agent stack that enables natural language control of desktop applications and web browsers through visual recognition and precise GUI interaction.

  • Project
    CopilotKit/openmuse

    A personal agent application with a persistent browser, Linux terminal, and Google integrations that allows users to delegate complex tasks and review agent actions.

  • Project
    getpaseo/paseo

    An open-source agentic development environment that provides a unified interface for multiple AI coding agents across desktop, mobile, and CLI.

  • Project
    spacering-net/codeg

    A multi-agent coding workspace that integrates various AI coding agents into a single visual interface, allowing them to collaborate and work on the same project.