Context-Aware Furigana: Solving the Ambiguity of Japanese Readings

Reading Japanese is a significant hurdle for learners because a single kanji character or compound can have multiple pronunciations depending on the context. While basic dictionary-based converters exist, they often fail on common words that change meaning and reading based on the surrounding text. EZFurigana aims to solve this by implementing a context-aware system that moves beyond simple lookup tables.

The Challenge of Japanese Homographs

For any student of Japanese, the frustration of encountering a word like "\u00a5\u00a5\u00a5" (market/stock market) is familiar. Depending on the sentence, it could be read as ichiba or shijou. Similarly, "\u00a5\u00a5\u00a5" (Oita prefecture/considerably) can be read as Oita or daibu.

These ambiguities make traditional furigana generators unreliable for learners who need accurate phonetic guides. To address this, the creator of EZFurigana developed a hybrid engine designed to handle these context-dependent target words.

The Technical Architecture: A Hybrid Approach

Rather than relying on a single model, EZFurigana uses a multi-layered system to determine the correct reading:

1. Sudachi for Tokenization

Sudachi serves as the foundation, providing tokenization, base forms, part-of-speech (POS) tagging, and initial candidate readings. This ensures the text is broken down into meaningful units before the reading is assigned.

2. Expanded Dictionaries and Custom Rules

To handle the nuances of the language, the system incorporates:

  • Expanded dictionary coverage for complex compounds and fixed expressions.
  • Custom rules specifically for counters, suffixes, and rendaku (sequential voicing) patterns.
  • Phrase overrides to handle common idioms that defy standard rules.

3. ModernBERT Fallback

For the most difficult cases—specifically 144 highly context-dependent target words—the system utilizes ModernBERT. This allows the engine to analyze the surrounding semantic context to choose the correct reading when dictionary rules and POS tags are insufficient.

Performance and Benchmarking

To ensure accuracy, the developer has been testing the system against an LLM-assisted benchmark of 7,500 Japanese lines. According to the author, the system currently averages about 12 wrong readings per 1,000 tokens.

While this is treated as a practical regression benchmark rather than a formal academic study, it allows the developer to continuously refine the engine and catch regressions in new versions. The most challenging remaining areas include personal names, rare vocabulary, domain-specific terms, and complex rendaku cases.

Community Feedback and Edge Cases

Early users on Hacker News have highlighted both the strengths and weaknesses of the current implementation. While some users were impressed by the tool's ability to correctly read difficult place names, others pointed out specific failures:

  • Contextual Misreads: One user noted that "\u00a5\u00a5\u00a5" (today) was read incorrectly in a sentence about Japanese society, where it should have been read as konnichi rather than the more common kyou.
  • Tokenization Errors: Another user observed that "\u00a5\u00a5\u00a5" (now what) was incorrectly read as konna ni (this much).
  • Reading Nuances: A user found that "\u00a5\u00a5\u00a5" (to what extent) was read as dou-hodo instead of ika-hodo.

These examples underscore the difficulty of the task; Japanese is a language where the "correct" reading is often tied to the subtle intent of the sentence.

Features for Learners

Beyond the technical engine, EZFurigana provides several utility features designed for the language learner's workflow:

  • Multi-format Support: The tool processes text, PDFs, images (via OCR), EPUB ebooks, and SRT subtitles.
  • JLPT Filtering: Users can filter furigana by JLPT level (N5-N1). For example, setting the filter to N3 will hide readings for easier N5/N4 kanji, forcing the learner to practice while providing a safety net for advanced vocabulary.
  • Export Options: Results can be exported as HTML, TXT, PDF, EPUB, SRT, or Anki-ready flashcards, integrating the tool directly into a spaced-repetition study system.

Sources