Doomformer: Compiling Integer Neural Networks into DOOM Maps

Overview

Doomformer is a specialized compiler that transforms integer neural networks into functional ZDoom maps. By mapping neural operations—such as arithmetic, activations, and attention mechanisms—to DOOM engine primitives like voodoo dolls, sector light levels, and floor heights, the project allows neural networks to execute within the game environment. The system validates its outputs by comparing every computed value against a NumPy integer reference with zero tolerance, ensuring exact parity between the game's simulation and the mathematical model.

How It Works

The core of Doomformer relies on the deterministic behavior of the DOOM engine, specifically ZDoom. The compiler maps network components to specific game elements:

  • Arithmetic: Performed by voodoo dolls (copies of the player) that interact with floor triggers and line specials.
  • Values: Stored as sector light levels or plane heights.
  • Activations: Represented as staircases or gate mechanisms.
  • Attention: Modeled as corridors of ceiling gates.

Python acts as the host, writing inputs to the map and reading back the final state. Because the system is designed for exactness, any deviation in the game's calculation compared to the NumPy reference results in a MismatchError. This allows developers to treat a DOOM map as a verifiable, albeit slow, execution engine for neural logic.

Performance and Optimization

Running neural networks inside a game engine is computationally expensive. A standard MLP execution in Doomformer is approximately 23,000 times slower than an equivalent NumPy implementation. The project introduces "optimization levels" (--opt) to manage this overhead:

  • Default Schedule: Designed for maximum compatibility and visual demonstration, often taking over 160 seconds for a single inference.
  • --opt 3: A more aggressive schedule that uses ACS (Action Code Script) to copy layer values between gates, significantly reducing the number of game tics required. For example, a handwritten digit inference can be reduced from 163 seconds to 8 seconds using this optimization.

Research and Practical Applications

Beyond simple demonstration, Doomformer supports complex architectures and research tasks:

  • Transformers: The toolkit can compile transformers, including models trained on DOOM level names or specific factual datasets.
  • Learning: The system supports perceptrons that can learn directly within the map, with weights updated by voodoo dolls based on input labels provided by Python.
  • Interventions: Users can perform activation patching and lesion studies within the game, observing how closing specific "neuron" doors affects the marine's behavior or the network's output.
  • Machine Simulation: The project includes a SUBLEQ machine and a counter machine, demonstrating that the map's geometry can function as a stored-program computer.

Community Insights

Hacker News users have expressed significant interest in the project's unique approach to neural network visualization. Commenters noted that while the project is "absolutely incredible," it requires a deep understanding of the DOOM engine to fully grasp the implementation details. Others highlighted the contrast between this approach and other attempts to run neural networks in unconventional environments, such as running DOOM inside a transformer model itself.

"I feel like this could help visualize a neural network, but you have to know a lot about DOOM to really understand it." — slowin

"Self-plug I have a very similar blog post title but a very different approach. Honestly, I like your idea more than my own idea." — stmonty

Technical Limitations

Doomformer is constrained by the engine's physical limits. The compiler must account for specific behaviors, such as plane mover limitations, object list caps (4,096 objects in ViZDoom), and coordinate wrapping in different ZDoom-family ports. The system is primarily validated in ViZDoom 1.3.2, and while many maps run in other ports like GZDoom or Zandronum, arithmetic behavior can vary slightly between engine versions.

Sources