Adolanium/hermes-gadget-sdk

What it solves

Hermes Gadget gives you a physical, button-press way to talk to an AI agent. Instead of pulling out a phone or laptop, you hold a TALK button, ask a question, and hear the reply from a small device on your desk or in your hand. It also lets the agent control the device’s LEDs and sensors, so the gadget becomes a tangible interface for your agent’s tools.

How it works

The project is an SDK plus ready-made firmware for supported boards. The device runs a portable C++17 core that handles the display, button input, and communication. It connects over Wi-Fi to a Hermes Agent (an LLM agent) using a WebSocket protocol. The agent’s plugin exposes the gadget’s controls as agent tools, and speech recognition / text‑to‑speech come from the agent’s own configuration. You can flash the firmware over USB from a browser installer, then manage Wi‑Fi from your phone. A desktop simulator runs the same core, so you can test the experience without any hardware. The Python tools handle building, simulating, and deploying the device.

Who it’s for

Hobbyists and makers who want to build a dedicated voice assistant for their own AI agent, and developers who want to extend the hardware—adding new boards, sensors, or custom faces. The simulator lowers the bar for trying out the interaction before committing to hardware, and the Home Assistant / MQTT examples show how to integrate the gadget into a smart home setup.

Highlights

• Runs on ESP32-S3 and Raspberry Pi 4/5, plus a desktop simulator for instant testing.
• Hold‑to‑talk interaction: press the button, speak, release, and hear the answer.
• The agent can read and control the gadget’s LEDs, sensors, and other controls as tools.
• Over‑the‑air firmware updates after the initial USB flash, with automatic rollback if a build cannot reach Hermes.
• Examples for Home Assistant and MQTT with a configured lamp and temperature sensor.
• MIT licensed, an independent community project.