bpwhelan/GameSentenceMiner

An immersion toolkit for learning Languages through games and other visual media.

What it solves

GSM (GameSentenceMiner) removes the friction from language learning while gaming. It automates the tedious process of manually creating flashcards, allowing you to capture game dialogue, audio, and visuals without needing to pause or leave your game window.

How it works

The application acts as a bridge between your game and Anki. It uses text-hooking tools or built-in OCR (Optical Character Recognition) to read game text directly from your screen. When you trigger a capture, it automatically records the associated audio, takes a screenshot or GIF, and sends the data to your flashcard deck. It also provides a transparent overlay for instant dictionary lookups using Yomitan.

Who it’s for

It is designed for language learners who use gaming as an immersion tool and want to build vocabulary through "sentence mining" without interrupting their gameplay flow.

Highlights

  • Automated Flashcards: Creates Anki cards with synchronized audio, screenshots, or GIFs.
  • Built-in OCR: Uses custom text-recognition technology to extract text from games that do not support standard text-hooking.
  • In-Game Overlay: Provides instant dictionary lookups and Furigana display without leaving the game.
  • Immersion Tracking: Includes a dashboard to track your reading progress, Kanji history, and daily goals.

Related

  • Project
    raine/anki-llm

    A scriptable CLI tool that uses LLMs and text-to-speech to bulk-process, generate, and improve Anki flashcards.

  • Project
    dominostars/playtranslate

    A real-time translation and language learning Android app that uses OCR to translate text from games, visual novels, and manga directly on the screen.

  • Project
    THUSI-Lab/GameScaling

    GameScaling is a set of vision-language models that were continued pre-training on 1,600 hours of keyboard-and-mouse gameplay so they can play computer games in real time. Each model takes the current screen (plus a short history) and outputs the next 200 ms of input as six 33 ms key steps plus one mouse move and scroll; the repo provides the inference client, system prompt template, examples, and tests for serving the released checkpoints.

  • Dispatch
    SubSmith Language Learning Platform – Overview and Community Feedback

    SubSmith lets users drop any video or audio file to generate subtitles, hover‑translate words, and export flashcards to Anki, aiming to make immersive language learning from personal media affordable and efficient.

  • Project
    thanhkeke97/RSTGameTranslation

    RSTGameTranslation is a Windows‑only tool that captures a game’s screen region, runs OCR, sends the text to an AI translator (Google, Gemini, Ollama, etc.), and draws the translation back onto the game. It also supports voice‑to‑text (Whisper/FunASR) and offline text‑to‑speech (Supertonic). Users select a window or rectangle, configure language and translator, and control everything via hotkeys. The app is useful for playing untranslated games, reading manga, or streaming with live subtitles.