pytorch/pytorch

Tensors and Dynamic neural networks in Python with strong GPU acceleration

What it solves

PyTorch is a machine learning framework that provides a flexible way to build and train deep neural networks. It solves the problem of needing a high-performance tensor library with GPU acceleration and a system for automatic differentiation (autograd) to optimize neural network weights.

How it works

PyTorch uses a "tape-based" autograd system (reverse-mode auto-differentiation), which allows for dynamic neural networks. Unlike static frameworks, PyTorch allows users to change the network's behavior on the fly without needing to rebuild the structure from scratch. It is deeply integrated with Python, allowing developers to use standard Python debuggers and libraries like NumPy and SciPy. For performance, it integrates acceleration libraries such as NVIDIA cuDNN, NCCL, and Intel MKL.

Who it’s for

PyTorch is designed for deep learning researchers who need maximum flexibility and speed for experimentation, as well as developers building production-ready AI models.

Highlights

  • GPU-Ready Tensors: Provides a NumPy-like tensor library that can run on CPUs or GPUs for massive acceleration.
  • Dynamic Computation Graphs: Uses a tape-based system that allows for arbitrary changes to network behavior with zero lag.
  • Python-First Design: Deeply integrated into Python, making it intuitive and compatible with the Python scientific stack.
  • Efficient Memory Management: Includes custom GPU memory allocators to enable the training of larger models.
  • Extensible: Offers a straightforward API for writing new layers in Python or C++.

Related

  • Project
    pyg-team/pytorch-frame

    A modular deep learning extension for PyTorch designed to build neural network models on heterogeneous tabular data containing mixed column types like text, images, and numerical values.

  • Project
    pytorch-tabular/pytorch_tabular

    A high-level PyTorch framework that provides a unified interface for applying various deep learning architectures to tabular data.

  • Project
    neuralhydrology/neuralhydrology

    A PyTorch-based Python library designed for deep learning research in hydrology, allowing users to train neural networks via configuration files.

  • Project
    Lightning-AI/pytorch-lightning

    A deep learning framework that abstracts away the engineering boilerplate of PyTorch, enabling researchers to pretrain and finetune AI models at scale across various hardware.

  • Project
    Ascend/pytorch

    TorchNPU is a PyTorch adapter plugin that enables the PyTorch framework to run on Huawei Ascend AI processors, providing optimized compute and distributed training capabilities.