HeyWillow/willow

Open source, local, and self-hosted Amazon Echo/Google Home competitive Voice Assistant alternative

What it solves

Willow is a platform for language inference tasks, providing capabilities for speech-to-text (STT), text-to-speech (TTS), and large language model (LLM) processing. It aims to provide fast inference for these modalities, including support for WebRTC applications.

How it works

Willow operates as an inference server that can be self-hosted, allowing users to run language-related AI tasks locally or on their own infrastructure to achieve high-speed performance.

Who it’s for

It is designed for early adopters and developers who want to host their own inference server for STT, TTS, and LLM tasks.

Highlights

  • Self-hostable inference server
  • Support for speech-to-text (STT)
  • Support for text-to-speech (TTS)
  • LLM inference capabilities
  • Compatible with WebRTC applications

Related

  • Project
    QuentinFuxa/WhisperLiveKit

    An ultra-low-latency, self-hosted speech-to-text pipeline that enables real-time transcription and translation using advanced simultaneous speech research and multiple model backends.

  • Project
    KoljaB/RealtimeTTS

    A Python text-to-speech library designed for low-latency audio generation from strings, generators, and LLM token streams, supporting a wide variety of local and cloud TTS engines.

  • Project
    yomorun/yomo

    An open-source LLM function calling framework for building scalable, ultra-fast AI agents using a geo-distributed architecture to reduce latency.

  • Project
    llm-d/llm-d

    llm‑d is an open‑source, CNCF‑sandbox project that adds production‑grade routing, KV‑cache tiering, and expert‑parallelism to existing LLM model servers (vLLM, SGLang). Delivered as Helm charts for Kubernetes, it lets organizations serve large language models at scale with lower latency and higher throughput across many accelerator types.

  • Project
    microsoft/LMOps

    A research initiative focused on fundamental technologies to improve LLM prompting, inference speed, and context handling for building AI products.