Jev Plays Pokémon Red – Live AI Gameplay Demo and Community Insights

Quick Take

Jev, an AI‑driven decision model, can play the entire Pokémon Red game in a live stream, demonstrating inexpensive, rapid action selection while also revealing notable failure modes such as looping in doors and sub‑optimal battle choices.

What Jev Does

Live gameplay: The demo streams Pokémon Red with audio muted by default; users can unmute to hear game sounds.
Decision panel: A right‑hand panel lists every AI decision and the probability (odds) assigned to each choice.
Guided navigation: The AI follows a pre‑computed guide that tells it where to go next, similar to a human walkthrough.
Open source: The code is available on GitHub (https://github.com/christianmat/jev-pokemon) and the live app is hosted at https://jev-pokemon.vercel.app/.

How the System Works

State extraction – The game state is read from the emulator memory (a "memhack"), providing precise information about location, inventory, and battle status.
Decision model – A lightweight language model (referred to as Jev) receives a textual description of the current state and a list of possible actions (e.g., "move north", "select menu option").
Probability scoring – The model returns a probability distribution over actions; the highest‑probability action is sent as a button press.
Guidance overlay – A separate guide module supplies high‑level waypoints (e.g., "go to Pewter City"), constraining the action space and preventing aimless wandering.
Loop detection – The system logs each decision; when the same state recurs, the model may repeat actions, leading to observed loops.

"The right panel shows every decision and Jev's odds." – Project description

Community Observations

Strengths Highlighted

Speed and cost: Commenters note the AI’s rapid decision rate and low monetary cost (≈ $1.65 for 38 hours of play).
Transparency: The live decision panel lets observers see the model’s confidence, which is rare in many AI game demos.
Proof of concept: Completing the game (or reaching the Elite Four) demonstrates that a compact decision engine can handle a full RPG loop.

"Great work! Super cool to see it do the whole game. I spent my fable budget building something similar this week but only drove it to Brock." – hummusFiend

Weaknesses and Failure Modes

Looping behavior: Users reported the AI getting stuck for minutes at locations like Rocket Hideout, repeatedly opening the same door.
Rail‑roaded choices: The decision set is heavily constrained by the guide, making the AI appear "dumb" when it cannot deviate from pre‑written paths.
Sub‑optimal battle tactics: The AI sometimes makes poor moves, such as teaching Charizard the non‑damage move Counter instead of a fire‑type attack.
Lack of vision: The system relies on memory reads rather than raw pixel input, limiting its generality to other games.

"I wish Jev took in images so we could do this generically for any game, without memhacks." – avaer

Architectural Questions

Control of character movement: The "Jev calls" counter only increments at menu or battle prompts, suggesting a separate script handles low‑level movement while Jev decides high‑level actions.
Goal source: The guide appears to provide the high‑level goal (e.g., "reach the Pokémon Center"), but the exact mechanism for generating these goals is not exposed in the demo.
Potential for larger models: Some commenters wonder if a bigger model like Fable could replace the guide and handle both high‑level planning and low‑level execution.

"Looking at the diagram in the gh repo, it looks like this is entirely Jev. Are there any examples of people having a big model like Fable handle high level goals?" – lwarfield

Why This Matters

Benchmark for "system‑one" AI: Games like Pokémon Red provide a controlled environment to test fast, reflex‑style decision making without the latency of vision‑to‑text pipelines.
Hybrid AI pipelines: The project illustrates a promising split—use a high‑level planner (potentially a large LLM) for goals, and a lightweight decision engine for rapid execution.
Cost‑effective research: Demonstrating a full‑game run for under two dollars shows that extensive AI gameplay experiments can be performed without massive compute budgets.

Open Questions & Future Directions

Vision‑based input: Replacing memory hacks with image‑to‑text models could broaden applicability to modern games.
Dynamic goal generation: Integrating a large LLM to generate and adapt goals on the fly may reduce reliance on handcrafted guides.
Loop mitigation: Implementing state‑history awareness or reinforcement‑learning style penalties could prevent endless door‑looping.
Multiplayer considerations: As noted by a community member, deploying such agents in online games raises anti‑cheat and fairness concerns.

Bottom line: Jev’s live Pokémon Red run proves that a compact decision model can navigate a complex RPG at low cost, but the demo also highlights the need for better state awareness, vision integration, and higher‑level planning to move beyond rail‑roaded, loop‑prone behavior.