pollen-robotics/reachy_mini_conversation_app

What it solves

This project provides a conversational interface for the Reachy Mini robot, enabling it to interact with humans using real-time voice, vision, and physical movement. It bridges the gap between high-level AI reasoning (LLMs) and physical robot hardware, allowing the robot to express emotions, dance, and perform tasks based on natural language conversations.

How it works

The app connects a user to a real-time AI backend (via Hugging Face) and the robot's hardware. It uses a layered motion system that blends speech-reactive movements (like wobbling) with queued primary actions (like dances or specific poses).

It employs a tool-based architecture where the AI assistant can trigger specific functions:

• Local Tools: Python modules that control the robot's hardware (e.g., moving the head, capturing camera frames).
• Remote Tools: MCP-compatible Hugging Face Spaces that provide external data (e.g., web search, weather, or time).

Users can define "profiles" (personalities) that include specific system prompts and sets of enabled tools to change how the robot behaves.

Who it’s for

• Robot owners and developers using the Reachy Mini hardware.
• AI researchers interested in embodied intelligence and human-robot interaction.
• Developers looking to build and test custom robot behaviors using LLM-based tool calling.

Highlights

• Real-time Interaction: Low-latency audio conversation and vision processing.
• Personality System: Customizable profiles with unique instructions and tool access.
• Hybrid Tooling: Supports both local Python tools for hardware control and remote MCP tools for external API access.
• Integrated Motion: Choreographed movements, including a library of dances and emotions, that blend with speech.
• Web UI: An optional interface for managing personalities, tools, and settings.