labmlai/annotated_deep_learning_paper_implementations

🧑‍🏫 60+ Implementations/tutorials of deep learning papers with side-by-side notes 📝; including transformers (original, xl, switch, feedback, vit, ...), optimizers (adam, adabelief, sophia, ...), gans(cyclegan, stylegan2, ...), 🎮 reinforcement learning (ppo, dqn), capsnet, distillation, ... 🧠

What it solves

Reading a deep learning paper and trying to implement it from scratch is hard. This project makes that easier by providing simple, working PyTorch code for many important papers, along with detailed notes that explain how each piece works. The notes are rendered side-by-side with the code on the website, so you can read the explanation next to the actual implementation.

How it works

You get a set of clean, minimal PyTorch implementations for a wide range of neural network models and algorithms. Each implementation is documented with explanations that describe the reasoning behind the design. The project is organized by topic, with folders for transformers, diffusion models, GANs, reinforcement learning, optimizers, normalization layers, and more. The website (nn.labml.ai) displays these notes and code together, making it easy to follow along. The repository is actively maintained, with new implementations added almost weekly.

Who it's for

Anyone who wants to understand deep learning papers by seeing them translated into simple, working code. This includes students, researchers, and practitioners who prefer learning by reading code and explanations rather than only reading math or papers.

Highlights

  • Simple PyTorch implementations, not production-heavy frameworks
  • Explanations are written for each implementation and rendered side-by-side with code
  • Covers a broad range of topics: transformers, diffusion models, GANs, reinforcement learning, optimizers, normalization layers, and more
  • Actively maintained, with new implementations added regularly
  • Includes an installation package (pip install labml-nn) for easy reuse
  • Also links to a separate skill for coding agents to implement any paper

Related

  • Project
    jingyaogong/minimind

    MiniMind is a lightweight, from-scratch LLM framework that enables individuals to train and experiment with language models on consumer hardware for minimal cost. It provides a complete educational pipeline covering pre-training, fine-tuning, and reinforcement learning without relying on complex third-party abstractions.

  • Project
    keras-team/keras

    Keras 3 is a multi-backend deep learning framework that allows you to build and train models using JAX, TensorFlow, or PyTorch, preventing framework lock-in while providing high-level usability and state-of-the-art performance.

  • Project
    scikit-learn/scikit-learn

    A Python module for machine learning built on top of SciPy, providing a wide range of tools and algorithms for ML practitioners.

  • Project
    josephmisiti/awesome-machine-learning

    A curated, language-organized list of machine learning frameworks, libraries, and tools, with extra resources and clear maintenance rules.