pytorch/pytorch

Tensors and Dynamic neural networks in Python with strong GPU acceleration

What it solves

PyTorch is a machine learning framework that provides a flexible way to build and train deep neural networks. It solves the problem of needing a high-performance tensor library with GPU acceleration and a system for automatic differentiation to handle the complex math required for training AI models.

How it works

PyTorch uses a "tape-based" autograd system (reverse-mode auto-differentiation), which allows developers to change the behavior of their neural networks dynamically during execution. Unlike static frameworks, it executes code imperatively, meaning lines of code are run as they are written. It integrates deeply with Python and leverages acceleration libraries like NVIDIA cuDNN, NCCL, and Intel MKL to maximize speed and memory efficiency on both CPUs and GPUs.

Who it’s for

It is designed for deep learning researchers who need maximum flexibility and speed for experimentation, as well as developers looking for a GPU-accelerated replacement for NumPy for scientific computation.

Highlights

  • GPU-Ready Tensors: Provides a NumPy-like tensor library that can run on CPUs or GPUs for massive acceleration.
  • Dynamic Neural Networks: Uses a tape-based autograd system for flexible, real-time network adjustments.
  • Python-First Design: Deeply integrated with Python, allowing the use of NumPy, SciPy, and Cython without reinventing the wheel.
  • Imperative Execution: Intuitive, linear execution that makes debugging straightforward with clear stack traces.
  • Efficient Memory Management: Includes custom GPU memory allocators to enable the training of larger models.

Related

  • Project
    pyg-team/pytorch-frame

    A modular deep learning extension for PyTorch designed to build neural network models on heterogeneous tabular data containing mixed column types like text, images, and numerical values.

  • Project
    pytorch-tabular/pytorch_tabular

    A high-level PyTorch framework that provides a unified interface for applying various deep learning architectures to tabular data.

  • Project
    neuralhydrology/neuralhydrology

    A PyTorch-based Python library designed for deep learning research in hydrology, allowing users to train neural networks via configuration files.

  • Project
    Lightning-AI/pytorch-lightning

    A deep learning framework that abstracts away the engineering boilerplate of PyTorch, enabling researchers to pretrain and finetune AI models at scale across various hardware.

  • Project
    facebookresearch/pytorch3d

    A PyTorch-based library for 3D Computer Vision research providing differentiable rendering and efficient tools for manipulating 3D meshes and point clouds.