reinforcement-learning: a collection of standalone PyTorch implementations of core reinforcement learning algorithms

What it solves

This repository provides a collection of clear, standalone code examples for reinforcement learning (RL) algorithms, ranging from basic concepts to deep reinforcement learning. It simplifies the learning process by providing one file per algorithm, making the implementations easy to read and study.

How it works

The project organizes RL algorithms into three main categories based on the environment they are used in:

• Grid World: Basic RL concepts like Policy Iteration, Value Iteration, SARSA, Q-Learning, and REINFORCE.
• CartPole: Implementations of Deep Q-Networks (DQN) and Proximal Policy Optimization (PPO) to solve the balance task.
• Atari: More advanced implementations of DQN and PPO applied to Atari games like Breakout and Montezuma's Revenge.

Who it’s for

It is designed for those looking for a practical, hands-on approach to learning reinforcement learning through readable code examples and benchmarked performance metrics on modern hardware (Apple Silicon).

Highlights

• Standalone Implementations: Each algorithm is contained in a single file for ease of reference.
• Citations: Each file includes the original paper citation and the core update equation.
• Modernized Stack: Built with PyTorch 2.11, Gymnasium 1.2, and Pygame for rendering.
• Performance Benchmarks: Includes detailed training times, RAM usage, and final scores for Atari games on M3/M4 chips.