pyg-team/pytorch-frame

Tabular Deep Learning Library for PyTorch

What it solves

PyTorch Frame provides a modular framework for building deep learning models specifically for heterogeneous tabular data. It addresses the limitations of traditional tree-based models (like GBDTs) by allowing for better integration with downstream models and the ability to handle complex column types such as text, images, and timestamps alongside numerical and categorical data.

How it works

The library uses a modular architecture consisting of three primary components:

  1. Materialization: Converts raw pandas DataFrames into a TensorFrame format suitable for PyTorch training.
  2. FeatureEncoder: Encodes various column types into hidden embeddings of a consistent size.
  3. TableConv: Models the interactions between these column embeddings.
  4. Decoder: Pools the embeddings to generate a final prediction or embedding per row.

It supports integration with LLM embedding APIs (such as OpenAI, Cohere, and Voyage AI) and Hugging Face transformers to encode text data.

Who it’s for

It is designed for deep learning researchers and practitioners who want to experiment with deep tabular models or compare their performance against GBDTs using a standardized, modular approach.

Highlights

  • Diverse Column Support: Handles numerical, categorical, multicategorical, text (embedded or tokenized), timestamps, and images.
  • Pre-implemented Models: Includes state-of-the-art models like Trompt, FTTransformer, TabNet, ExcelFormer, and TabTransformer.
  • PyTorch Ecosystem Integration: Works seamlessly with other PyTorch libraries, including PyG for relational database learning.
  • Benchmark Datasets: Provides a collection of ready-to-use tabular datasets and benchmarking tools to compare deep learning models against GBDTs.

Related

  • Project
    pytorch-tabular/pytorch_tabular

    A high-level PyTorch framework that provides a unified interface for applying various deep learning architectures to tabular data.

  • Project
    pytorch/pytorch

    A Python-based machine learning framework that provides GPU-accelerated tensor computation and a dynamic autograd system for building deep neural networks.

  • Project
    neuralhydrology/neuralhydrology

    A PyTorch-based Python library designed for deep learning research in hydrology, allowing users to train neural networks via configuration files.

  • Project
    rllm-team/rllm

    A PyTorch library for Relational Table Learning (RTL) that standardizes and combines GNNs, LLMs, and TNNs to analyze data across linked tables.

  • Project
    facebookresearch/multimodal

    A PyTorch library for training state-of-the-art multimodal models at scale, providing modular building blocks and pretrained weights for models like CLIP and BLIP-2.