HorizonWind2004/reconstruction-alignment

[ICLR 2026] RecA: visual understanding help generation through self-supervised learning

What it solves

RecA (Reconstruction Alignment) addresses the gap between visual understanding and image generation in unified multimodal models (UMMs). It aims to improve the quality of text-to-image generation and image editing without requiring expensive image-specific captions or reinforcement learning.

How it works

RecA is a self-supervised post-training method. It trains a pretrained UMM to reconstruct images from its own internal visual understanding features, using the original images as the target. This process aligns the model's generative capabilities with its semantic understanding, which then transfers to better performance in generation and editing tasks while maintaining the original inference interface.

Who it’s for

Researchers and developers working with unified multimodal models who want to enhance image generation and editing capabilities through self-supervised alignment.

Highlights

  • Architecture Agnostic: Verified on multiple architectures including BAGEL, Harmon, OpenUni, and Show-o.
  • Efficient Training: For example, BAGEL can be improved on generation and editing benchmarks in approximately 4.5 hours using 6 x 80GB A100s.
  • Self-Supervised: Uses original images as targets instead of manual captions.
  • Performance Gains: Demonstrates significant improvements across benchmarks like GenEval and DPGBench across various model sizes (0.5B to 14B parameters).

Related

  • Project
    google-research/augmix

    AugMix is a data processing technique that mixes augmented images and enforces consistent embeddings to improve the robustness and and uncertainty calibration of image classification models.

  • Project
    ShuangLI59/unified_video_action

    A unified model for robotic manipulation that combines video generation and action prediction through a two-stage training process to improve visual foresight and task execution.

  • Project
    SkyworkAI/UniPic

    A unified multimodal model series for image editing, generation, and understanding, featuring high-speed multi-image editing and autoregressive modeling.

  • Project
    krea-ai/krea-2

    Krea 2 is an open-source image generation model featuring a base RAW model for flexible fine-tuning and a distilled Turbo model for fast, high-quality text-to-image synthesis.

  • Dispatch
    Using LoRA for Efficient Stable Diffusion Fine-Tuning

    Hugging Face has integrated Low-Rank Adaptation (LoRA) into the diffusers library, enabling faster Stable Diffusion fine-tuning with significantly lower VRAM requirements and model weights as small as 3 MB.