hmorimitsu/ptlflow

PyTorch Lightning Optical Flow models, scripts, and pretrained weights.

What it solves

It provides a unified framework for estimating optical flow, allowing users to train and test multiple state-of-the-art deep learning models within a single, consistent environment rather than using fragmented original implementations.

How it works

Built on PyTorch Lightning, the project integrates a wide collection of optical flow models (such as RAFT, FlowFormer, and GMFlow) and supports various industry-standard datasets (including Middlebury, VIPER, and Spring). It includes tools for inference, validation, and benchmarking model speed and accuracy.

Who it’s for

Researchers and developers working on computer vision, specifically those focusing on motion estimation and dense correspondence between video frames.

Highlights

  • Extensive Model Library: Includes dozens of state-of-the-art models like Flow-Anything, FlowSeek, and NeuFlow.
  • Unified Interface: Simplifies the process of training and testing different architectures using a consistent API.
  • Dataset Support: Integrated support for multiple benchmarks including TartanAir, Kubric, and Monkaa.
  • Optimization: Supports FP16 precision for most models to improve performance and
  • Deployment Tools: Provides ONNX and TensorRT conversion for specific models like RAPIDFlow.

Related

  • Project
    Lightning-AI/pytorch-lightning

    A deep learning framework that abstracts away the engineering boilerplate of PyTorch, enabling researchers to pretrain and finetune AI models at scale across various hardware.

  • Project
    sktime/pytorch-forecasting

    A PyTorch-based library for time series forecasting that provides a high-level API and state-of-the-art deep learning architectures like Temporal Fusion Transformers and N-BEATS.

  • Project
    X-GenGroup/Flow-Factory

    A unified framework for applying reinforcement learning to fine-tune diffusion and flow-matching models across image, video, and audio-video tasks.

  • Project
    microsoft/onnxruntime

    A cross-platform machine learning accelerator that optimizes inference and training for models from various frameworks like PyTorch and TensorFlow.

  • Project
    pytorch-tabular/pytorch_tabular

    A high-level PyTorch framework that provides a unified interface for applying various deep learning architectures to tabular data.