espressif/esp-dl

What it solves

ESP-DL is designed to enable the deployment of neural network models on resource-constrained ESP series chips. It provides a lightweight inference framework that allows developers to run AI applications directly on-device, avoiding the need for constant cloud connectivity for basic AI tasks.

How it works

The framework uses a custom lightweight model format (.espdl) based on FlatBuffers for zero-copy deserialization. It integrates with ESP-PPQ, a quantization tool that converts models from ONNX, PyTorch, and TensorFlow into the optimized ESP-DL format. To maximize performance on ESP32 series chips, it employs several hardware-specific optimizations:

• Static Memory Planner: Automatically allocates layers to optimal memory locations based on available internal RAM.
• Dual Core Scheduling: Distributes computationally heavy operators (like Conv2D) across dual-core processors to increase speed.
• 8-bit LUT Activation: Uses Look Up Tables for activation functions to reduce computational complexity.
• Efficient Operators: Implements optimized versions of common AI operators such as Conv, Gemm, Add, and Mul.

Who it’s for

Embedded developers and AI engineers who are building AI-powered IoT devices using Espressif's System on Chips (SoCs).

Highlights

• Broad Model Support: Supports quantization and deployment of models like YOLO11n, YOLO26, and PP-OCRv6.
• Seamless Integration: Works directly with ESP-IDF and integrates with the ESP-PPQ quantization tool.
• Hardware-Optimized: Specifically tuned for ESP32-S3 and ESP32-P4 chips for maximum efficiency.
• AI Agent Tooling: Includes tools to help AI agents automatically implement and optimize operators for the framework.