isair/jarvis

What it solves

Jarvis provides a private, local-first voice assistant that acts as a "third person in the room." It eliminates the need for cloud subscriptions and constant typing by allowing users to have natural, voice-driven conversations that can incorporate the context of ongoing discussions with other people.

How it works

The system runs speech recognition, language models, and speech synthesis on the user's own hardware. It maintains a temporary rolling transcript of nearby speech to understand the context of a conversation before being addressed. For long-term memory, it uses local databases with semantic search indices and a knowledge graph. It integrates with local model servers like Ollama, LM Studio, or llama.cpp, and can extend its capabilities via MCP servers for browser automation and smart-home control.

Who it’s for

It is designed for users who want a powerful AI assistant but prioritize privacy and local control over cloud-based services, as well as those who prefer voice-first interaction over traditional chat interfaces.

Highlights

• Local-First Privacy: Speech recognition and LLMs run on your hardware; conversation memory and diaries are stored locally.
• Conversational Context: Can join ongoing discussions by tracking recent speech, allowing it to answer questions based on what was just said.
• Extensible Tooling: Includes built-in tools for web search, OCR, and nutrition tracking, with support for MCP servers to add custom integrations.
• Hybrid Interface: Features an animated desktop presence for voice interaction and a companion chat window for silent typing.
• Cross-Platform: Available for macOS, Windows, and Linux.