OHF-Voice/speech-to-phrase

Fast and personalized local speech-to-text

Speech‑to‑Phrase – a Home‑Assistant‑focused speech recogniser

What it is – A locally‑run speech‑to‑text system that doesn’t try to transcribe arbitrary speech. Instead it answers the question “Which of the known command phrases did the user say?” by matching spoken input against a set of templates that include your Home Assistant entities, areas and floors.

Why it exists – For home‑automation you usually only need to recognise a limited, well‑defined set of commands (turn on the kitchen light, add apples to the shopping list, etc.). By restricting the vocabulary to those phrases the recogniser can be fast, run offline and be personalised to the exact names you use in Home Assistant.

How it works (high‑level)

  1. Sentence templates – Pre‑written language files (YAML) describe the commands you can say. The repository ships templates for many languages and you can add your own.
  2. Dynamic expansion – At start‑up the tool merges the templates with the names of entities, areas and floors that you have exposed to Home Assistant, producing a hassil template file that represents every possible utterance.
  3. Finite‑State Transducer (FST) – Using the Rhasspy white‑paper techniques, the compact template is compiled into an FST (via OpenFST and OpenGRM). This efficiently encodes the huge combinatorial space without enumerating every sentence.
  4. Kaldi acoustic model – The FST is turned into a language model for Kaldi, an open‑source speech‑recognition toolkit. A built‑in pronunciation dictionary and a Phonetisaurus model supply phonemes for known and unknown words.
  5. Post‑processing fuzzy layer – After Kaldi produces a raw transcription, a small fuzzy‑matching step cleans up minor errors (duplicate/missing words) and forces entity names to match exactly what Home Assistant knows.

Key features

  • Local‑only – No cloud service, works behind your firewall.
  • Multi‑language – Kaldi‑based version (v1.4.3) supports 22 languages; a newer rewrite (in OHF‑Voice/apps) currently covers the most common ones.
  • Custom sentences – Add your own YAML files to extend the command set without touching the code.
  • Docker ready – Official container (rhasspy/wyoming-speech-to-phrase) that can be hooked to Home Assistant via the Wyoming integration.
  • Automatic model download – Acoustic models and pronunciation tools are fetched from the public HuggingFace dataset rhasspy/rhasspy-speech.
  • Home Assistant integration – Connects through the Home Assistant websocket, using a long‑lived access token.

Getting started (quick steps)

  1. Install – Either add the add‑on from Home Assistant’s Supervisor store or run the Docker command shown in the README.
  2. Provide credentials – Supply --hass-websocket-uri and --hass-token so the service can read your entities.
  3. (Optional) Custom sentences – Create a directory with language‑specific sentences.yaml files and launch with --custom-sentences-dir /path/to/dir.
  4. Run – The service will download the required acoustic model, build the FST, train the Kaldi language model and start listening on port 10300.
  5. Use – Send audio to the Wyoming endpoint; the recogniser returns the matched phrase, which Home Assistant can then turn into an automation.

Languages currently supported (Kaldi v1.4.3)

English, French, German, Dutch, Spanish, Italian, Russian, Czech, Catalan, Greek, Romanian, Portuguese (Portugal), Polish, Hindi, Basque, Finnish, Mongolian, Slovenian, Swahili, Thai, Turkish.

Extending / customizing

  • Add new entity names – Expose devices to Home Assistant’s Assist feature; they are automatically incorporated.
  • Create new command templates – Follow the format in speech_to_phrase/sentences/<lang>/sentences.yaml or supply your own via the custom‑sentences directory.
  • Add new languages – Contribute a pronunciation dictionary and acoustic model (or reuse an existing Kaldi model) and add sentence templates; see issue #149 for the current status.

Where the project is now

The Kaldi‑based code in this repository is frozen at version 1.4.3. Development has moved to the OHF‑Voice/apps monorepo, where the system is being rewritten to use a constrained CTC decoder, Home‑Assistant‑intents for sentence sources, and a web UI for selecting commands.

Who might use this

  • Home‑automation enthusiasts who want offline, privacy‑preserving voice control.
  • Developers building custom Home Assistant voice assistants and needing a recogniser that can understand the exact entity names they expose.
  • Anyone looking for a lightweight alternative to cloud‑based speech‑to‑text when the command set is bounded.

Links

  • Add‑on badge – Quick install from Home Assistant Supervisor.
  • Docker image – rhasspy/wyoming-speech-to-phrase.
  • Home Assistant intents repo – Source of sentence templates for the new rewrite.
  • Rhasspy whitepaper – Underlying techniques for FST compilation.
  • Issue #149 – Tracker for language support and migration details.

Related

  • Project
    OHF-Voice/wyoming-faster-whisper

    Wyoming Faster Whisper is a server that implements the Wyoming protocol for Home Assistant, using the faster‑whisper speech‑to‑text engine. It offers fast, local transcription, optional GPU support, VAD‑based endpointing, and can bias results with Home Assistant entity names.

  • Project
    Picovoice/leopard

    Leopard is an on-device speech-to-text engine that provides private, accurate, and computationally efficient transcription across multiple platforms.

  • Project
    fishaudio/fish-speech

    Fish Speech is an open‑source, large‑model‑powered text‑to‑speech system offering expressive, multilingual voice cloning, multi‑speaker scene generation, and on‑the‑fly audio editing. It provides two models – Drama 3 (35 B) for long‑form, high‑fidelity drama production, and S2‑Pro (4 B) for faster, consumer‑GPU deployment – with command‑line, API, and Web UI interfaces, Docker support, and detailed installation guides.

  • Project
    OHF-Voice/linux-voice-assistant

    A Linux-based voice satellite for Home Assistant that runs on x64 or ARM64 hardware, enabling custom smart speakers with local wake word detection and seamless integration via the ESPHome protocol. It handles on-device wake word recognition and provides flexibility for additional features, all while connecting to Home Assistant's Assist voice system.

  • Project
    k2-fsa/sherpa-onnx

    A versatile, cross-platform audio AI toolkit that enables local execution of speech-to-text, text-to-speech, and speaker analysis models on everything from mobile devices to embedded hardware.