akdeb/ElatoAI

Realtime Voice AI with 100+ Models on Arduino ESP32 with Secure Websockets and Edge Functions for AI Companions, and Devices

What it solves

ElatoAI enables the creation of real-time voice AI devices using affordable ESP32 hardware. It bridges the gap between high-level AI speech models and low-power IoT devices, allowing developers to build physical objects (like AI toys) that can engage in long, low-latency, natural conversations without requiring expensive on-board processing power.

How it works

The system uses a three-tier architecture to handle the heavy lifting of AI processing while keeping the hardware lightweight:

  1. ESP32 IoT Client: Acts as the audio interface, capturing user speech and playing back AI responses using the Opus audio codec for high-quality, low-bandwidth streaming.
  2. Edge Server Functions: Deployed on Deno Edge or Cloudflare Workers, these servers manage secure WebSocket connections and route audio data between the ESP32 and various AI providers.
  3. AI Model Providers: The edge server connects to frontier speech-to-speech APIs such as OpenAI Realtime, Gemini Live, xAI Grok, Eleven Labs, and Hume AI to generate responses.

Who it’s for

It is designed for IoT developers, hardware hackers, and creators who want to integrate advanced voice AI into physical products or DIY electronics projects using the Arduino framework.

Highlights

  • Broad Model Support: Compatible with over 100 voice AI models via FastAPI and native integrations for top-tier real-time APIs.
  • Optimized Audio: Uses Opus compression to maintain high clarity at 12kbps and a 48KHz sampling rate.
  • Low Latency: Achieves a 2-3 second round-trip global latency using edge computing.
  • Hardware Friendly: Does not require PSRAM on the ESP32 to run speech-to-speech AI.
  • Comprehensive Tooling: Includes a Next.js web app for device management, volume control, and AI personality customization.

Related

  • Project
    hackers365/xiaozhi-esp32-server-golang

    A high-performance Go-based AI backend for ESP32 devices that enables low-latency, full-streaming voice interactions by integrating ASR, LLM, and TTS engines.

  • Project
    xinnan-tech/xiaozhi-esp32-server

    A backend server for ESP32-based smart hardware that enables custom voice-controlled AI interactions, device management, and multi-modal processing.

  • Project
    joey-zhou/xiaozhi-esp32-server-java

    A Java-based enterprise backend for ESP32 smart hardware, providing a management platform and AI dialogue services with RAG, voiceprint recognition, and multi-LLM support.

  • Dispatch
    OpenAI GPT-Live: Engineering a Real-time Voice AI System

    OpenAI has introduced GPT-Live, a third-generation voice system that utilizes a full-duplex voice model and a new low-latency architecture to enable continuous, natural voice interaction without the need for turn detectors.

  • Project
    espressif/esp-claw

    An AI agent framework for ESP32-series IoT devices that enables users to define device behavior through chat and execute decision-making locally on the edge.