tinygrad/tinygrad

You like pytorch? You like micrograd? You love tinygrad! ❤️

What it solves

tinygrad is a lightweight, end-to-end deep learning stack designed to be a middle ground between the simplicity of micrograd and the full-featured nature of PyTorch. It provides a hackable, transparent compiler and IR (Intermediate Representation) that allows users to see and modify how kernels are generated and executed on various hardware accelerators.

How it works

It combines a PyTorch-like eager Tensor API with an IR-based autodiff system inspired by JAX. The framework uses a "laziness" approach to fuse multiple operations into a single kernel to optimize performance. It includes a JIT (Just-In-Time) compiler, graph execution, and a full set of neural network primitives (nn, optim, datasets) for training real models.

Who it’s for

Researchers, developers, and students who want a deep learning framework that is easy to read, hackable, and transparent, rather than a complex "black box" industrial framework.

Highlights

  • End-to-End Stack: Includes everything from the tensor library and autograd to the IR, compiler, and neural network layers.
  • Broad Accelerator Support: Supports a wide range of backends including CUDA, Metal, OpenCL, CPU, AMD, NV, QCOM, and WebGPU.
  • Transparent Compiler: The entire compiler and IR are visible, making it easier to actually understand how deep learning operations are lowered to hardware.
  • linhas-efficient: Intentionally kept small and readable to reduce complexity.

Related

  • Project
    karpathy/micrograd

    A tiny, educational autograd engine that implements backpropagation over a scalar-based DAG to help users understand the internal mechanics of neural networks.

  • Project
    pytorch/pytorch

    A Python-based machine learning framework that provides GPU-accelerated tensor computation and a dynamic autograd system for building deep neural networks.

  • Project
    pytorch-tabular/pytorch_tabular

    A high-level PyTorch framework that provides a unified interface for applying various deep learning architectures to tabular data.

  • Project
    BobMcDear/attorch

    A Triton-based subset of PyTorch's nn module that provides a readable and hackable collection of neural network layers for developing custom deep learning operations.

  • Project
    Farama-Foundation/Minigrid

    A collection of lightweight, discrete grid-world environments for Reinforcement Learning research, supporting goal-oriented tasks and grounded language learning.