kyutai-labs/hibiki

Hibiki is a model for streaming speech translation (also known as simultaneous translation). Unlike offline translation—where one waits for the end of the source utterance to start translating--- Hibiki adapts its flow to accumulate just enough context to produce a correct translation in real-time, chunk by chunk.

What it solves

Hibiki provides high-fidelity, simultaneous speech-to-speech translation. Unlike traditional offline translation, which requires the entire utterance to be finished before translating, Hibiki translates in real-time, chunk by chunk, as the user speaks.

How it works

It uses a decoder-only model with a multistream architecture (inherited from Moshi) to jointly model source and target speech. The model produces text and audio tokens at a constant rate of 12.5Hz, allowing for a continuous output stream. It was trained on synthetic data generated via a contextual alignment method that ensures target words only appear once they are predictable from the source.

Who it’s for

Developers and researchers interested in real-time audio translation, specifically for French-to-English translation, and those looking for on-device deployment on smartphones (via Hibiki-M).

Highlights

  • Simultaneous Translation: Generates target speech and text translation in real-time as the input is processed.
  • Voice Transfer: Optionally maintains the original speaker's voice characteristics in the translated output.
  • on-device capability: Available in 1B and 2B parameter versions, with the 1B model designed for local execution on smartphone hardware.
  • Cross-platform support: Provides inference code for PyTorch, Rust, MLX (macOS), and MLX-Swift (iOS).

Related

  • Project
    azuwis/pianotrans

    A GUI and packaging tool for ByteDance's piano transcription system that converts piano audio and video recordings into MIDI files with pedal data.

  • Project
    espressif/esp-sr

    A speech recognition framework for Espressif chips that enables offline wake word detection, voice command recognition, and audio preprocessing.

  • Project
    alchaincyf/huashu-skills

    A curated index of 53 Claude‑compatible *Agent Skills* (design, data, video, writing, persona‑based reasoning, etc.) with install instructions, an update checker, and a machine‑readable catalog. Enables AI agents to perform concrete productivity tasks without custom coding.

  • Project
    abundantbeing/hermes-browser-extension

    A Chromium side‑panel extension that connects your browser to a Hermes Agent (local, cloud, or remote). It streams active‑tab context, supports multi‑agent switching, smart tab scoping, site‑aware drafting (Hermes Assist), page comments, and optional live tab control, all with transparent diagnostics and nine visual themes.