sosoj92/jarvis-assistant-vocal

What it solves

Jarvis is a local voice assistant designed to act as a central hub for home automation, PC control, and personal productivity. It eliminates the need for multiple fragmented apps by integrating voice-activated control over hardware (lights, OBS, PC stats), web services (Gmail, Google Calendar, Instagram), and complex workflows (web reservations, content creation pipelines) into a single, natural-language interface.

How it works

Jarvis uses a modular pipeline: a local wake word detector (openWakeWord) triggers a local transcription engine (faster-whisper), which sends the request to a configurable LLM. The system supports three operational modes: Hybrid (cloud for daily tasks, local for background), Quality (high-power cloud models), and Local (100% offline via Ollama).

To handle complex reasoning without compromising security, Jarvis delegates deep research and thinking to Hermes, a separate local deliberative agent. Jarvis retains the "body" (execution of actions and API keys), while Hermes provides the "brain" (analysis and drafting). The system can be expanded via network audio satellites (e.g., Raspberry Pi) to provide voice access in different rooms without long cables.

Who it’s for

Users who want a highly customizable, privacy-conscious voice assistant that can control their Windows 11 environment and smart home, as well as developers looking for an extensible framework to add their own tools via a simple decorator-based system.

Highlights

• Multi-modal Control: Integrates voice, screen vision (capturing errors/text), and webcam-based hand gestures for PC and media control.
• Extensive Tooling: Built-in integrations for Philips Hue, OBS, Google Calendar, Gmail, Discord, Instagram, and Twilio for phone calls.
• Web Automation: Uses Playwright and Chrome CDP to summarize active tabs, manage browser tabs, and perform real-world web reservations.
• Graduated Security: Implements N1/N2/N3 permission levels to ensure critical actions (like sending emails or shutting down the PC) require explicit vocal confirmation.
• Flexible Deployment: Can run on a single PC or a headless server with a remote "main station" for audio and display.
• Local-First Options: Supports completely offline operation using Ollama and local TTS engines like Piper or Kokoro.