lucidrains/autoregressive-diffusion-pytorch

What it solves

This project implements a method for generating images without needing vector quantization (VQ), which is a common requirement in many autoregressive image models. It allows for the generation of image data treated as a sequence of tokens using a diffusion-based approach.

How it works

The library provides a PyTorch implementation of the architecture described in the paper "Autoregressive Image Generation without Vector Quantization." It treats images as sequences of patches (tokens) and applies an autoregressive diffusion process to generate them. It also includes an improvised version using flow matching (via ImageAutoregressiveFlow) for alternative generation paths.

Who it’s for

Researchers and developers working on generative AI, specifically those interested in image synthesis and alternative architectures to traditional VQ-based autoregressive models.

Highlights

• Supports image generation without vector quantization.
• Includes a dedicated ImageTrainer for simplifying the training process on image datasets.
• Provides both standard autoregressive diffusion and flow-matching versions.
• Supports explorative modeling, allowing training against the best of multiple candidates per timestep.