espressif/esp-sr

ESP‑SR – Speech‑Recognition Framework for ESP‑Series MCUs

What it is

• An open‑source software library from Espressif that bundles ready‑made AI models and signal‑processing blocks for on‑device speech interfaces.
• Targets the low‑power ESP32 family (ESP32, ESP32‑S2/S3/S31, ESP32‑P4, ESP32‑C3/C5/C6, etc.).

Core components

Module	Function	Typical models
Audio Front‑End (AFE)	Echo cancellation, voice‑activity detection, blind source separation, noise suppression (including deep‑learning‑based NSNET)	Built‑in algorithms, configurable for 1‑mic or 2‑mic setups
Wake‑Word Engine (WakeNet)	Always‑on detection of a trigger phrase (e.g., “Hi, ESP”)	WakeNet9, WakeNet9s (low‑resource), WakeNet10 (newer, INT16‑quantized)
VADNet	Stand‑alone voice‑activity detection, can replace WebRTC VAD	VADNet model released 2025
Speech‑Command Recognition (MultiNet)	Offline keyword/command spotting for up to 300 commands	Chinese/English models mn5‑mn7, scaled for different ESP chips
Speech Synthesis	Text‑to‑speech pipeline (supports several languages)	TTS‑trained WakeNet models, V3 pipeline adds more languages

Why it matters

• Edge AI: All inference runs on the MCU, no cloud required, keeping latency low and preserving privacy.
• Tiny footprint: Models are quantized (INT8/INT16) and tuned for the limited RAM/CPU of ESP32 chips.
• Modular: Each block is a reusable component that can be mixed‑and‑matched in ESP‑IDF projects.
• Multi‑language support: Wake‑word and TTS pipelines cover Chinese, English, Japanese, French and are expanding to many more languages.

Typical use cases

• Voice‑controlled smart home devices (lights, thermostats, appliances) built on ESP32‑S3 or ESP32‑P4.
• Battery‑powered wearables or IoT sensors that need a local “Hey …” trigger.
• Embedded robots or hobbyist projects that require on‑device command spotting without a server.
• Prototyping of voice assistants that must run offline for privacy or connectivity‑limited environments.

Getting started

1. Install the ESP‑IDF development environment.
2. Add the esp-sr component via the Espressif Component Registry (badge in the README).
3. Choose the desired model (e.g., wn9_hilexin for a wake‑word) and include the corresponding header in your application.
4. Use the provided examples in the esp‑skainet repository to flash a demo onto an ESP32‑S3 board.
5. Consult the online documentation for model migration, AFE configuration, and custom wake‑word training.

Documentation & resources

• Full API docs: https://docs.espressif.com/projects/esp-sr/en/latest/esp32s3/index.html
• Migration guide from V1 → V2: https://docs.espressif.com/projects/esp-sr/en/latest/esp32s3/audio_front_end/migration_guide.html
• Wake‑word customization process: https://docs.espressif.com/projects/esp-sr/en/latest/esp32s3/wake_word_engine/ESP_Wake_Words_Customization.html
• Example projects: https://github.com/espressif/esp-skainet/tree/master/examples
• Related DSP components: esp-dl/audio (Fbank/MFCC) and dl_fft.

License

• The repository is released under the Apache‑2.0 License (standard for Espressif SDKs), allowing free use in both open‑source and commercial products provided attribution is kept.

---

Espressif’s ESP‑SR turns a modest microcontroller into a fully‑featured speech‑enabled device, handling everything from raw microphone input to wake‑word detection, command recognition, and even text‑to‑speech, all on‑chip.