deepgenteam/deepgen

What it solves

DeepGen 1.0 is a lightweight unified multimodal model designed to handle multiple image-related tasks within a single framework. It addresses the challenge of needing massive model scaling to achieve high performance in image generation, editing, and reasoning, providing a more efficient alternative to much larger state-of-the-art models.

How it works

DeepGen 1.0 combines a 3B parameter Vision Language Model (VLM) and a 2B parameter Diffusion Transformer (DiT). It uses a deep alignment framework called Stacked Channel Bridging (SCB) to extract hierarchical features from the VLM and fuse them with "think tokens" to guide the generative backbone.

The model is trained in three stages:

1. Alignment Pre-training: Synchronizes VLM and DiT representations using large-scale image-text pairs and editing triplets.
2. Joint Supervised Fine-tuning (SFT): Trains on a mixture of generation, editing, and reasoning tasks.
3. Reinforcement Learning (RL): Uses MR-GRPO with a mixture of reward functions to improve generation quality and human preference alignment.

Who it’s for

This project is for researchers and developers working on multimodal AI, specifically those looking for a lightweight yet powerful model for text-to-image generation, image editing, and reasoning-based image manipulation.

Highlights

• Unified Capabilities: Supports general image generation, general image editing, reasoning image generation, reasoning image editing, and text rendering in one model.
• Lightweight Architecture: Only 5B total parameters (3B VLM + 2B DiT), yet competitive with models 3x to 16x larger.
• Reasoning-Rich Guidance: Employs Stacked Channel Bridging (SCB) for better semantic understanding and fine-grained control.
• Diffusers Compatible: Provides a format compatible with the Hugging Face Diffusers library for easy integration.