kevinhughes27/TensorKart

What it solves

TensorKart enables an AI agent to drive Mario Kart 64 autonomously by learning from human driving samples. It addresses the challenge of mapping visual game state (screenshots) to specific controller inputs to navigate race tracks.

How it works

The project uses a behavioral cloning approach where a user records their own gameplay using a joystick and a screen-capture tool. These image-input and joystick-output pairs are used to train a model via TensorFlow and cuDNN. Once trained, the model is deployed in a gym-mupen64plus environment, where it processes real-time screenshots from the emulator to predict and execute the necessary joystick commands.

Who it’s for

Developers and AI enthusiasts interested in imitation learning, autonomous vehicle concepts applied to gaming, and TensorFlow implementations.

Highlights

• Generalization: The model can potentially drive on tracks it wasn't trained on, even with small datasets.
• Human-in-the-loop: Includes tools for recording and viewing training samples.
• GPU Acceleration: Utilizes cuDNN for faster training times.
• Manual Override: Allows a human player to override AI commands by holding the 'LB' button during playback.