FatihMakes/Mark-LVI

Mark LVI – A Cross‑Platform Personal AI Assistant

What it is – Mark LVI is a desktop‑level AI companion that can listen, speak, see your screen/webcam and control the operating system. It runs on Windows, macOS and Linux and can use either Google’s Gemini Live API (the default) or any self‑hosted or third‑party language model that implements the standard /v1/chat/completions endpoint (Ollama, LM Studio, llama.cpp, vLLM, OpenAI, Groq, DeepSeek, Mistral, xAI, OpenRouter, etc.).

Key capabilities

• Real‑time voice interaction – push‑to‑talk (Ctrl+Space), wake‑word (“Hey Jarvis”), ultra‑low latency streaming via Gemini Live or local Whisper + Piper pipelines.
• Holographic avatar – an animated head with lip‑sync, facial expressions and status cues, rendered in software (no GPU required).
• Multilingual – Whisper detects the spoken language; Piper (or ElevenLabs) replies in the same language, supporting ~50 languages with 176 voices.
• Full system control – launch/close apps, adjust volume/brightness, Wi‑Fi, power, clipboard utilities, file processing, web‑search, browser control, messaging (WhatsApp, Telegram), hardware monitoring, weather, reminders, etc.
• Plugin & skill system – drop a single .py file into plugins/ and the assistant auto‑discovers a new tool; each tool declares its own TOOL dict.
• Memory & undo – persistent, searchable memory of facts, per‑session summarisation, and an “undo” that reverts file or setting changes.
• Safety features – self‑echo guard, confirmation gate for irreversible actions, explicit done/unverified/failed status reporting, untrusted‑content boundary, and per‑device volume handling.
• Fallback engine ladder – nine Gemini models are ordered; if quota, timeout or outage occurs the system automatically switches to the next model without breaking the conversation.
• Local‑first option – you can run the whole stack locally (Whisper for STT, Piper for TTS, any local LLM) with no subscription cost; the UI only loads providers when you enable them (0 ms overhead when disabled).

How it works

1. Engine selection – via the UI you pick a provider for hearing, thinking and speaking. Each part is tested with a one‑click “TEST” button that reports connectivity and latency.
2. Conversation flow – audio is streamed to the chosen model; the model can call tools (e.g., open_app, read_file). Tool results are fed back to the model, which then replies to you.
3. Tool budgeting – only the most relevant tools are sent to the model (default 20 tools, size‑limited) to keep payloads small and latency low.
4. Session continuity – the conversation survives device changes, drops, or voice switches; context is compressed with a sliding‑window algorithm for unlimited session length.

Installation & first‑run

• Clone the repo and run the provided installer script (or pip install -r requirements.txt).
• The base install only pulls google‑genai and httpx; optional components (Whisper, Piper, ElevenLabs) are installed on demand from the UI.
• On first launch the assistant registers itself to start on boot (registry/LaunchAgent/.desktop).
• Pair a phone dashboard via QR code if you want remote control.

Why it matters Mark LVI demonstrates a fully self‑contained, extensible AI agent that bridges voice, vision and OS automation while giving users the freedom to run everything locally or on any cloud LLM. Its modular provider architecture, safety‑first tool handling and rich UI make it a practical example of a personal “Jarvis‑style” assistant that can be customized without writing code.

---

All details are taken directly from the project’s README; no external information has been added.