bquenin/interpreter

Interpreter – Offline screen translator for Japanese retro games

What it is – A desktop application that captures the image of any window (typically a retro Japanese game), runs OCR on the captured frames, translates the extracted Japanese text to English, and shows the result as subtitles. All processing happens locally; no cloud services are used.

Key capabilities

• Fully offline – After the initial model download (≈1.5 GB) the program works without an internet connection, keeping your data private.
• OCR – Uses MeikiOCR, a model trained on pixel‑perfect Japanese game fonts, to recognise text from low‑resolution screenshots.
• Translation – Employs the Sugoi V4 Japanese‑to‑English model (via CTranslate2) to produce English subtitles.
• Two overlay styles –
  • Banner mode: a draggable subtitle bar at the bottom of the screen.
  • In‑place mode: transparent text rendered directly over the original Japanese characters, click‑through so the game remains playable.
• Caching – Fuzzy‑matching of previously seen strings avoids re‑translating identical or similar text, improving speed.
• Multi‑display support – The overlay appears on the same monitor as the captured window.

Supported platforms

• Windows 10 (1903+)
• macOS
• Linux (X11, XWayland, Wayland) – with extra notes for Wayland (global hotkeys, PipeWire capture, fullscreen‑only in‑place overlay).

Installation

• One‑liner scripts for macOS/Linux (curl … | bash) and Windows PowerShell (irm … | iex).
• The installer pulls the binary interpreter‑v2, required libraries, and the OCR/translation models.

Running the program

interpreter-v2

Launches a GUI where you select the target window, choose overlay mode, adjust OCR confidence, and tweak other settings.

How it works internally

1. Screen capture – grabs frames from the chosen window at a configurable rate.
2. OCR – feeds each frame to MeikiOCR to obtain Japanese text and its screen coordinates.
3. Translation – passes the extracted strings to the Sugoi V4 model (hosted on Hugging Face) via CTranslate2 for fast CPU/GPU inference.
4. Display – renders the English text either in a bottom banner or directly over the original glyphs.

Troubleshooting tips from the README

• Poor OCR: adjust the confidence slider; lower values capture more text but may add noise.
• Slow performance: the first run downloads the models; subsequent runs use the cached copy in ~/.cache/huggingface/.

Why it matters – It demonstrates a practical, privacy‑preserving AI pipeline (capture → OCR → neural‑machine translation) applied to a niche but popular use‑case: playing Japanese retro games without needing external translation services.