espressif/esp-brookesia

What it solves

ESP-Brookesia is a full-stack development platform designed to simplify the creation of AIoT (AI + IoT) and HMI (Human-Machine Interface) products. It addresses the complexity of integrating hardware abstraction, AI capabilities, and user interfaces on embedded devices by providing a modular, component-based architecture.

How it works

The platform organizes its functionality into ESP-IDF components across several layers:

• Hardware Abstraction (HAL): Unifies device access and supports both real ESP devices and PC simulation to decouple apps from specific hardware.
• Service Framework: Manages service registration, function calls, and event publishing, including a bridge for LLM invocation via MCP (Model Context Protocol).
• Runtime and GUI: Supports multiple runtimes (ELF, JS, Lua, WASM) and a JSON-driven GUI model (implemented via LVGL) for declarative interface design.
• System Framework: Integrates the GUI, runtimes, and services into a product-level system with app lifecycle management.
• AI Integration: Provides an agent manager and specific adapters for AI agents from OpenAI, Coze, and XiaoZhi, as well as AI-driven emote and animation management for visual feedback.

Who it’s for

It is intended for developers building smart home devices, anthropomorphic AI assistants, and other embedded AI products that require a combination of screen-based interaction and AI agent integration.

Highlights

• Multi-Runtime Support: Ability to run apps in JavaScript, Lua, WebAssembly, or ELF.
• AI Agent Ecosystem: Built-in adapters for major AI platforms and an agent manager for lifecycle and state management.
• MCP Bridge: A utility that allows LLMs to invoke Brookesia service functions as tools.
• PC Simulation: Reduces hardware dependency during development through Linux and WebAssembly simulation.
• Declarative GUI: Uses a JSON UI model to separate interface design from backend logic.