Shmuma/ptan

PTAN – PyTorch AgentNet

What it is – PTAN is a lightweight Python library that re‑implements the AgentNet framework for reinforcement‑learning (RL) agents using PyTorch. It provides helper classes (e.g., experience buffers, training loops) that were used in the Deep Reinforcement Learning Hands‑On book.

Key components

• ExperienceSource and ExperienceSourceFirstLast – utilities for collecting trajectories from OpenAI‑Gym environments.
• Integration with PyTorch Ignite for easy training loops.
• Optional TensorBoardX logging and OpenCV support for visualisation.

Supported environments – Any environment compatible with OpenAI Gym, including Atari games (gym[atari]).

Installation

# From PyPI (recommended)
pip install ptan

# Or directly from the repository
pip install git+https://github.com/Shmuma/ptan.git

# From source
python setup.py install

Dependencies (as listed in the README)

• PyTorch ≥ 1.1.0 (tested up to 1.7 on the master branch)
• PyTorch‑Ignite
• OpenAI Gym (plus the optional atari extras)
• opencv‑python
• tensorboardX (or tensorboard-pytorch for Anaconda users)

Versioning – The repository maintains separate branches for different major PyTorch versions:

• master – latest tested PyTorch (currently 1.7)
• torch-1.3-book-ed2 – code matching the second‑edition book, built for PyTorch 1.3
• torch-1.7 – merged into master

Documentation – A Jupyter notebook introduction is provided at docs/intro.ipynb. For a deeper dive into the experience‑source utilities, see the issue comment linked in the README.

Typical workflow (high‑level, inferred from the README)

1. Install the library and its dependencies.
2. Create a Gym environment.
3. Use ExperienceSource to generate batches of (state, action, reward, next_state) tuples.
4. Feed those batches to a PyTorch model inside an Ignite trainer.
5. Log metrics with TensorBoardX.

Who it’s for – Developers and researchers who want a ready‑made scaffolding for building and training RL agents in PyTorch without writing boiler‑plate code.