jitsi/jiwer

Evaluate your speech-to-text system with similarity measures such as word error rate (WER)

What it solves

It provides a fast and simple way to evaluate the accuracy of automatic speech recognition (ASR) systems by comparing the predicted text (hypothesis) against a ground-truth reference.

How it works

The library computes the minimum-edit distance between reference and hypothesis sentences. It uses RapidFuzz (a C++ based implementation) to ensure high performance, avoiding the slowness of pure Python implementations.

Who it’s for

Developers and researchers working on speech-to-text models who need to measure transcription accuracy using standard industry metrics.

Highlights

  • Supports multiple evaluation measures: Word Error Rate (WER), Match Error Rate (MER), Word Information Lost (WIL), Word Information Preserved (WIP), and Character Error Rate (CER).
  • Optimized for speed using C++ under the hood via RapidFuzz.
  • Handles empty reference strings to allow testing for model hallucinations on silent audio.