espressif/esp-dl

What it solves

ESP-DL provides a lightweight neural network inference framework specifically for Espressif ESP series chips. It allows developers to deploy AI models on resource-constrained System on Chips (SoCs) by optimizing memory usage and execution speed.

How it works

The framework uses a custom .espdl model format based on FlatBuffers for zero-copy deserialization. It integrates with ESP-PPQ, a quantization tool that converts models from ONNX, PyTorch, and TensorFlow into the ESP-DL format. To maximize performance on ESP hardware, it employs:

• Mixed Quantization: Supports 8-bit, 16-bit, and mixed-precision (w8a16) to balance accuracy and speed.
• Static Memory Planning: Automatically allocates layers to the best memory locations based on available internal RAM.
• Hardware Acceleration: Utilizes dual-core scheduling for intensive operators like Conv2D and uses 8-bit Look Up Tables (LUT) for most activation functions.

Who it’s for

Developers building AI applications for ESP32 and other Espressif SoCs who need to run pre-trained models on-device.

Highlights

• Broad Model Support: Includes a Model Zoo with pre-trained models for tasks like OCR (PP-OCRv6) and object detection (YOLO11n, YOLO26).
• Efficient Quantization: Features tools like AutoQuant and the espdl-quantize skill to automatically find optimal quantization strategies.
• Netron Visualization: Supports visualizing .espdl models for easier debugging.
• Agent-Ready Tooling: Includes tools to help AI agents automatically implement and optimize operators.