PixelCNN++: Improving PixelCNN with discretized logistic mixture likelihood and other modifications

TL;DR

OpenAI announced PixelCNN++, an improved PixelCNN model that uses a discretized logistic mixture likelihood, whole‑pixel conditioning, downsampling, shortcut connections, and dropout, and reports state‑of‑the‑art log likelihood on CIFAR‑10.

Overview

PixelCNN++ is a generative model for images that provides an exact likelihood and was released by OpenAI on January 19, 2017. The release includes the source code on GitHub and a paper describing the modifications.

Key Technical Modifications

The modifications in PixelCNN++ improve both training speed and model performance. First, the model replaces the 256‑way softmax over pixel values with a discretized logistic mixture likelihood, which speeds up training. Second, it conditions on whole pixels rather than individual R/G/B sub‑pixels, simplifying the architecture. Third, downsampling layers are added to capture multi‑resolution structure efficiently. Fourth, additional shortcut connections are introduced to accelerate optimization. Fifth, dropout is applied as a regularizer.

Experimental Results

The authors evaluated PixelCNN++ on the CIFAR‑10 dataset and report that it achieves state‑of‑the‑art log likelihood, demonstrating the effectiveness of the proposed changes.

Implications

By showing that a simple likelihood change and architectural tweaks can improve PixelCNN performance, PixelCNN++ provides a practical recipe for building stronger likelihood‑based image generators and encourages further research into scalable autoregressive models.