cjpais/Handy

A free, open source, and extensible speech-to-text application that works completely offline.

Handy – Offline, privacy‑first speech‑to‑text

What it is – Handy is a free, open‑source desktop app that lets you dictate into any window and have the text appear instantly, all without sending audio to the cloud. It runs locally on Windows, macOS, and Linux and uses on‑device neural models (OpenAI‑style Whisper or the CPU‑optimized Parakeet) for transcription.

How it works

Press a configurable global shortcut (hold‑to‑record or toggle).
Speak – audio is captured via cpal and filtered with Silero VAD.
When you release the shortcut, the audio is fed to the selected model (Whisper Small/Medium/Turbo/Large or Parakeet V3).
The resulting text is automatically pasted into the currently focused application.

All steps happen on your machine; no network traffic is required.

Key Features

Offline inference – works with Whisper GGML/GGUF or Parakeet models; GPU acceleration is used when available.
Cross‑platform – native builds for macOS (Intel & Apple Silicon), Windows x64, and Linux x64.
Extensible – built with Tauri (Rust backend, React/TS frontend) so developers can add plugins or new models.
Global hot‑keys – rdev provides system‑wide shortcuts; CLI flags let you control a running instance (handy --toggle-transcription).
Voice‑activity detection – Silero VAD filters silence before transcription, reducing unnecessary processing.
Raycast integration – macOS users can control Handy from the Raycast launcher.
Debug mode & CLI – built‑in debug overlay, verbose logging, and many start‑up flags for power users.

Architecture at a glance

Layer	Technology	Role
UI	React + TypeScript, Tailwind CSS	Settings window, overlay UI
App shell	Tauri (Rust)	Cross‑platform packaging, system integration
Audio I/O	`cpal`	Capture microphone streams
VAD	`vad‑rs` (Silero)	Detect speech vs. silence
Hot‑keys	`rdev`	Global keyboard shortcuts
Resampling	`rubato`	Convert audio to model‑required sample rate
Whisper inference	`transcribe‑cpp` (GGML/GGUF)	GPU‑or‑CPU Whisper models
Parakeet inference	`transcribe‑rs`	CPU‑optimized Parakeet V3 model

Getting started

Install – download a binary from the GitHub releases page, or use a package manager:
- macOS: brew install --cask handy
- Windows: winget install cjpais.Handy
Run the app, grant microphone & accessibility permissions, and set a shortcut in Settings → Keyboard.
Press the shortcut, speak, release – the transcription is pasted where your cursor is.

Advanced control – use the CLI flags to toggle recording, start hidden, disable the tray icon, etc. Example:

handy --start-hidden --no-tray &   # launch silently
handy --toggle-transcription       # toggle from a custom hot‑key or script

Known limitations & troubleshooting

Bluetooth mic on macOS – may lower playback volume; use the built‑in mic or a wired one for best results.
fn/Globe key shortcuts work only on Apple keyboards due to a hardware HID limitation.
Whisper model crashes on some Windows/Linux configurations – community help is requested.
Linux Wayland – requires wtype or dotool for text insertion; overlay can steal focus, so disabling it (Overlay Position: None) is recommended.
Runtime library – on Linux you may need libgtk-layer-shell0 (or the distro‑specific package) to start the app.

Extending Handy

Because the backend is Rust and the UI is a standard web stack, contributors can:

Add new model loaders (e.g., other GGUF/ONNX models).
Improve global shortcut handling across OSes.
Build additional integrations (e.g., a VS Code extension or a browser plugin).
Refactor settings storage or add a plugin system for custom post‑processing.

Roadmap (as of the README)

Debug logging to file.
Better macOS shortcut support (Globe key, rewrite of global shortcut handling).
Optional opt‑in analytics (privacy‑first).
Settings system cleanup and Tauri command refactoring.

Quick sanity check for power users

Handy signs its release artifacts with Tauri’s updater signature. The public key lives in src-tauri/tauri.conf.json; you can verify a download with minisign as shown in the README.

Bottom line – Handy provides a straightforward, fully offline dictation workflow that respects privacy, runs on the three major desktop OSes, and is built on open‑source speech‑recognition libraries. It’s a solid base for anyone who needs a forkable, extensible speech‑to‑text tool.