akdeb/ElatoAI

Realtime Voice AI with 100+ Models on Arduino ESP32 with Secure Websockets and Edge Functions for AI Toys, Companions, and Devices

What it solves

ElatoAI enables the creation of real-time voice AI agents on low-cost ESP32 hardware. It bridges the gap between powerful cloud-based speech-to-speech models and embedded IoT devices, allowing for low-latency, uninterrupted conversations (up to 20 minutes) without requiring expensive on-device processing power (PSRAM).

How it works

The system uses a three-tier architecture:

  1. ESP32 IoT Client: Captures audio and sends it via secure WebSockets using the Opus audio codec for high clarity and low bandwidth.
  2. Edge Server Functions: Deployed on Deno Edge or Cloudflare Workers, these act as a gateway that manages WebSocket connections and communicates with various AI providers (such as OpenAI, Gemini, xAI, Eleven Labs, and Hume AI).
  3. Frontend Client: A Next.js web application used to configure AI agents, manage devices, and monitor conversation transcripts.

Who it’s for

  • Hardware enthusiasts and DIY makers wanting to build AI-powered toys or gadgets.
  • Developers interested in integrating real-time voice AI into embedded systems.
  • Engineers exploring the deployment of voice pipelines using edge computing.

Highlights

  • Multi-Model Support: Integrates with OpenAI Realtime, Gemini Live, xAI Grok, Eleven Labs, and Hume AI EVI-4.
  • Low Latency: Achieves 2-3 second round-trip latency globally via edge deployment.
  • Efficient Audio: Uses Opus compression (12kbps) to maintain high audio quality on limited hardware.
  • Advanced IoT Features: Includes OTA (Over-the-Air) updates, WiFi management via captive portal, and tool calling capabilities.
  • Hardware Optimized: Designed to work on ESP32-S3 without requiring PSRAM.