stormydayy/halo-overlay

Lightweight opensource AI overlay for live conversations. Hears, sees, suggests.

Halo is a small AI assistant that floats as an overlay on your screen. It listens to the conversation around you, watches what's on your display, and gives you a short, real-time suggestion for what to say or do next.

What it does

Listens to two audio channels: the other side (system audio, labeled "Them") and you (microphone, labeled "You"). In "Both" mode, your microphone lines are suppressed while the other side is speaking, so your name isn't attached to their words.
Watches your screen continuously. It takes a tiny grayscale snapshot every few milliseconds, compares it to the previous one, and only keeps frames when the screen actually changes and then settles. This avoids storing duplicates or mid-animation frames.
Suggests a short answer when it detects a question in the other side's speech (auto-answer, off by default because it's a heuristic and can misfire).
Rewind (Ctrl+Shift+Enter): if you zoned out, it takes the transcript from the last 30 seconds to 5 minutes plus a few screen keyframes, and replies with three things: what happened (2–3 bullets), what they need from you now, and a ready-made line to say. You can also pin frames in the Rewind panel to include them in your next question.
Records sessions: every session is saved (transcript + suggestions). A Recap button has the AI write a summary with action items, and you can export everything to Markdown.
Profiles: interview, meeting, sales, negotiation, presentation, study — each with its own prompt style.
Customizable: shortcuts, window opacity, font size, and click-through mode are all adjustable.
Optional screen recording straight to disk as .webm (off by default).

How it's built

A from-scratch rewrite of the idea behind sohzm/cheating-daddy, but with no bundler, no vendored libraries, no build step — the source runs as-is.
One OpenAI-compatible client works with Gemini, Groq, OpenAI, OpenRouter, Ollama, LM Studio, or your own server. No separate code paths per provider.
Speech recognition uses VAD (voice activity detection) plus any Whisper-compatible API (Groq, OpenAI, or a local whisper.cpp / faster-whisper server).
~3,100 lines of code, 0 runtime dependencies, 27 unit tests (npm test).
Security: contextIsolation, sandbox, strict CSP, every IPC call validated, API keys encrypted by the OS (DPAPI / Keychain / libsecret) and never sent to the UI.

Privacy

API keys are encrypted with your OS's secure storage and never shown in the UI.
Data only goes to the providers you configure: short audio clips to the speech recognition service, and text (plus screenshots if enabled) to the model. Nothing phones home.
Sessions and recordings are stored locally in the app's profile folder. "Delete all sessions and recordings" or Ctrl+Shift+E erases them.
Screen recording is off by default.

Setup

Requires Node.js 20+ and Python 3.11+.

git clone https://github.com/stormydayy/halo-overlay
cd halo-overlay
npm install
npm start

Then in Settings: pick a model provider and paste your key, test the connection, choose a speech recognition service, pick a profile, add context (resume, notes, etc.), and start a session.

Fully local option: use Ollama with a vision-capable model (e.g. llama3.2-vision) for the LLM, and any server exposing /v1/audio/transcriptions (whisper.cpp, faster-whisper-server) for speech. Local addresses (localhost, 192.168.*, 10.*) don't need an API key.

Platform notes

Windows: system audio via loopback through getDisplayMedia; microphone works.
macOS: system audio via a SystemAudioDump helper (built for Apple Silicon only — Intel Macs need to build it yourself from source); microphone works. Grant Screen Recording and Microphone access in System Settings.
Linux: system audio via loopback if your environment supports it; otherwise microphone only.

If system audio isn't available, Halo tells you and continues with microphone + screen.

Known limitations

Speech recognition works in chunks split at pauses (typically 1–3 second delay), not as a continuous stream.
The question detector is a heuristic, so false positives happen — that's why auto-answer is off by default.
SystemAudioDump is built for Apple Silicon only.

License

GPL-3.0. The idea and the macOS audio helper come from sohzm/cheating-daddy and Mohammed-Yasin-Mulla/Sound.

Disclaimer (from the README)

Halo is a conversation aid, like a stage prompter. Rules vary: many interviews, exams, employers, and platforms prohibit outside help or recording, and laws on recording conversations differ by country. Check the rules wherever you use it, and tell the other people involved if the law or basic ethics call for it.