OpenAI OT-GAN: Improving GANs using optimal transport

OpenAI has introduced Optimal Transport GAN (OT-GAN), a generative adversarial network variant designed to minimize a new distance metric between the generator distribution and the data distribution. This approach improves training stability, particularly when using large mini-batches, and achieves state-of-the-art results on several image generation benchmark problems.

The Mini-Batch Energy Distance Metric

OT-GAN utilizes a metric called mini-batch energy distance. This distance function is designed to be highly discriminative and provides unbiased mini-batch gradients during training. It achieves this by combining two primary technical components:

• Optimal Transport in Primal Form: The metric incorporates optimal transport principles to measure the distance between distributions.
• Adversarially Learned Feature Space: The energy distance is defined within a feature space that is learned adversarially, allowing the model to better distinguish between real and generated data.

Training Stability and Performance

OT-GAN demonstrates significant improvements in stability and performance over traditional GAN architectures. The researchers found that OT-GAN is highly stable when trained with large mini-batches, which typically presents challenges for standard generative adversarial networks.

According to the the authors' abstract, the result of this architecture is a "highly discriminative distance function with unbiased mini-batch gradients."

Implications for Image Generation

By applying the mini-batch energy distance metric, OT-GAN achieves state-of-the-art results on popular benchmark problems for image generation. This suggests that the mini-batch energy distance approach provides a more effective way to align the generator's distribution with the target data distribution than previous GAN loss functions.