ciddwd/overlay-translator

📱 Screen Translator (屏译)

What it is – An Android app that captures the screen, runs OCR on the image, sends the recognized text to a translation engine (LLM‑based, cloud MT, or on‑device models) and overlays the translated result back onto the screen. It can also read the text aloud via TTS. The whole pipeline (capture → OCR → translate → display / speak) is modular, so users can mix‑and‑match local or remote services.

Key capabilities

• Real‑time overlay translation – a floating ball lets you translate the whole screen or a selected region with a single tap. The translation is rendered either directly over the original text or in a separate draggable floating window.
• Batch image translation – select multiple screenshots from the gallery (or share images to the app) and have them processed in the background.
• Multiple OCR back‑ends – on‑device ML Kit, PaddleOCR (v5/v6), a specialised manga OCR, or HTTP‑based services such as Umi‑OCR/LunaTranslator. Cloud OCR (Baidu, Tencent, Youdao, PaddleOCR AI Studio) is also supported as a fallback.
• Pluggable translators –
  • On‑device models: Sakura, Hy‑MT2 (both ~1 GB, run on CPU).
  • Cloud/LLM services: OpenAI‑compatible, Anthropic‑compatible, DeepSeek, Claude, ModelScope, OpenRouter, etc.
  • Traditional MT: DeepL / DeepLX, Google, Baidu, Tencent, Youdao, 小牛翻译.
• Text‑to‑speech – use the phone’s built‑in voice, a self‑hosted TTS server, or commercial services (火山, MiniMax, MiMo). The app can read original text, translation, or dictionary entries.
• Word‑level lookup – draw a box around a word/phrase to get a dictionary card that includes translation, pronunciation, part‑of‑speech, examples and, when an LLM is used, richer explanations.
• Smart UI – automatic detection of horizontal/vertical layout, bubble detection for manga, colour‑matching to the background, and optional “tight‑fit” rendering that erases the original text before drawing the translation.
• Customization – colour picker, font import (.ttf), per‑app translation presets, term‑bank for consistent names, export/import of non‑sensitive settings, encrypted storage of API keys, and a searchable settings page.
• Accessibility & stability – TalkBack labels, crash logs with sanitisation, background download of models with notification progress, and handling for aggressive OEM background‑kill policies.

Typical workflow

1. Install the APK (arm64‑v8a only) and grant overlay & notification permissions.
2. Start the screen‑capture service (MediaProjection or Shizuku for a smoother path).
3. While playing a game, reading a visual novel, or viewing a comic, tap the floating ball.
4. The app captures the screen, runs the selected OCR engine, sends the text to the chosen translator, and draws the result on‑screen.
5. Optional: long‑press the ball for a radial menu, use volume‑button shortcuts, or invoke the app from the Android quick‑settings tile.

Who benefits

• Gamers who need live subtitles for foreign‑language dialogue.
• Manga / visual‑novel readers who want on‑the‑fly translation of vertical Japanese text.
• Anyone who wants a privacy‑preserving, possibly fully offline, screen‑translation tool on Android.

Installation notes

• Download the latest ScreenTranslator‑x.y.z.apk from the Releases page (only arm64‑v8a builds are provided).
• Verify the SHA‑256 checksum if desired.
• After installation, enable the floating‑window permission and, if you want Shizuku integration, install the Shizuku app and grant it root‑less privileges.

Why it matters Screen Translator combines several frontier AI components—mobile OCR, on‑device large‑language‑model translation, and flexible TTS—into a single, user‑controllable Android experience. Its modular design lets power users keep everything on‑device for privacy, while casual users can point the app at cloud services for higher quality translations.