BasedHardware/omi

AI that sees your screen, listens to your conversations and tells you what to do

What it solves

Omi acts as a "second brain" by automatically capturing, transcribing, and remembering everything a user sees and hears across various devices. It solves the problem of forgetting important details from conversations or screen activity by providing a searchable, AI-powered memory of a user's daily digital and physical interactions.

How it works

Omi uses a cross-platform ecosystem to capture data in real-time:

  • Capture: It collects audio and screen data via a dedicated wearable device, a macOS app, or a mobile app.
  • Processing: Data is sent to a Python-based backend that handles Voice Activity Detection (VAD), diarization (identifying different speakers), and speech-to-text (STT) via Deepgram.
  • Storage & Retrieval: Information is stored in Firestore and cached with Redis, allowing LLMs to access this history to generate summaries, action items, and answer questions in a chat interface.
  • Hardware: The system includes open-source firmware for wearables (nRF/Zephyr) and glasses (ESP32-S3) that communicate via BLE.

Who it’s for

  • Professionals who need to track action items and summaries from numerous meetings and conversations.
  • Developers interested in building AI-powered wearable hardware or integrating custom AI personas.
  • Users who want a comprehensive, searchable archive of their life's audio and visual inputs.

Highlights

  • Multi-device support: Works across wearables, macOS, iOS, and Android.
  • Full-stack open source: Includes everything from hardware designs and firmware to the backend API and frontend apps.
  • Real-time capabilities: Features real-time transcription and audio streaming pipelines.
  • Extensible: Provides SDKs for multiple languages and an MCP (Model Context Protocol) server for integration.

Related

  • Project
    VictorTaelin/OptMem

    A permanent memory system for AI agents that uses a binary tree of summaries and a CLI tool to maintain persistent knowledge across sessions.

  • Project
    Notely-Voice/NotelyVoice

    A private, cross-platform AI voice transcription and note-taking app that uses OpenAI Whisper to provide offline speech-to-text capabilities.

  • Project
    melandlabs/openloomi

    An open-source AI attention agent that connects fragmented work tools and monitors screen context to surface critical reminders and maintain a holistic memory of projects and decisions.

  • Project
    aayoawoyemi/Ori-Mnemos

    An open-source persistent memory infrastructure for AI agents that uses a local knowledge graph of markdown files to implement cognitive-inspired retrieval and learning.

  • Project
    Nexus-JPF/note-companion

    An Obsidian plugin that uses AI to transcribe audio and YouTube videos, chat with vault context, and automatically organize messy notes into structured files.