BasedHardware/omi

AI that sees your screen, listens to your conversations and tells you what to do

📚 Quick Overview

Omi is an open‑source “second brain” platform that records what you see and hear, turns it into text in real‑time, and lets an AI chat remember all that context. It works across macOS, Windows, iOS/Android phones and a dedicated wearable device (Omi Glass). The system is split into:

Device firmware (nRF‑based wearables, ESP32‑S3 glass) that captures audio/video and streams it via BLE.
Desktop & mobile apps (Swift/macOS, Flutter for phones) that display the feed and host a chat UI.
Backend (Python/FastAPI) that runs speech‑to‑text (Deepgram), voice activity detection, diarisation, stores data in Firestore/Redis and forwards it to LLMs for summarisation and chat.

The repo contains the full stack – firmware, apps, backend, SDKs and docs – and can be run locally or against the public cloud backend.

🛠️ Core Components (from the README)

Component	Location	Tech Stack
macOS app	`desktop/macos/`	Swift, SwiftUI, Python desktop backend
Windows app	`desktop/windows/`	Node.js (Electron?)
Mobile app	`app/`	Flutter (iOS & Android)
Backend API	`backend/`	Python, FastAPI, Firebase, Redis
Firmware	`omi/`	nRF, Zephyr, C
Omi Glass	`omiGlass/`	ESP32‑S3, C
SDKs	`sdks/`	Python, Swift, React‑Native, multi‑lang BLE protocol
AI Personas (web)	`web/personas-open-source/`	Next.js

🚀 Getting Started (quick‑start commands)

macOS (single‑command demo)

git clone https://github.com/BasedHardware/omi.git && cd omi/desktop/macos && ./run.sh --yolo

Builds the macOS app, connects to the public backend and launches – no local env files needed.

Windows (dev mode)

git clone https://github.com/BasedHardware/omi.git
cd omi\desktop\windows
npm install
copy .env.example .env
npm run dev

Runs the Windows desktop app using the public config.

Full local development (backend, firmware, etc.)

make setup                     # installs git hooks & syncs Python env
python3 scripts/worktree_env_doctor.py --all --fix   # fixes missing deps
# then follow the steps in the README to build macOS app or flash firmware

📖 Documentation & Resources

Intro & quick‑start: https://docs.omi.me/quickstart
Backend deep‑dive: https://docs.omi.me/doc/developer/backend/backend_deepdive
BLE protocol spec: https://docs.omi.me/doc/developer/Protocol
SDKs: sdks/python/, sdks/swift/, sdks/react-native/
Hardware guides: assembly, buying, flashing – all on the docs site.

🎯 Who Might Use This?

Power users who want an AI‑augmented memory that records meetings, lectures, or daily life.
Developers looking to build custom AI‑powered wearables or integrate the BLE protocol into their own devices.
Researchers interested in end‑to‑end pipelines: capture → STT → diarisation → LLM summarisation.

⚖️ License

MIT – you can freely use, modify, and redistribute the code and hardware designs.

All information above is taken directly from the repository’s README; no external assumptions have been added.