bpwhelan/GameSentenceMiner

What it solves

GameSentenceMiner (GSM) simplifies the process of "sentence mining"—creating language learning flashcards from games. It removes the friction of manually capturing text, audio, and images from gameplay to import into Anki, allowing users to learn a language through immersion in gaming.

How it works

GSM acts as a bridge between a game and Anki. It uses several methods to extract text:

• Text Hooking: Integrates with tools like Agent or Textractor to pull text directly from the game's memory.
• OCR (Optical Character Recognition): For games without text hooks, it uses a custom fork of OwOCR to read text directly from the screen via customizable capture zones.
• Context Capture: It uses Voice Activity Detection (VAD) to automatically record and trim audio lines and takes screenshots (or GIFs) of the game state at the moment a line is spoken.
• Overlay: A transparent overlay allows users to to perform instant dictionary lookups via Yomitan without leaving the game window.

Who it’s for

Language learners who play video games in their target language to build vocabulary and acquire language through context-based flashcards.

Highlights

• Automated Anki Integration: Automatically adds audio, screenshots, and GIFs to flashcards.
• Integrated OCR: Built-in OCR for games that don't support text hooks.
• VAD-powered Audio: Uses Voice Activity Detection to capture specific voice lines.
• Immersion Tracking: A statistics dashboard with a Kanji grid and daily reading targets.
• AI Translation: Optional integration for sentence translations via API keys.