aimagelab/multimodal-garment-designer

What it solves

This project addresses the challenge of fashion image editing by moving beyond simple virtual try-ons. It enables designers to generate human-centric fashion images based on multimodal prompts, allowing them to communicate a design vision from concept to realization by controlling how clothes interact with the human body.

How it works

The system uses a human-centric latent diffusion model architecture. It generates images by following multimodal inputs, which can include text descriptions, human body poses, and garment sketches. To support this, the authors extended the Dress Code and VITON-HD datasets with semi-automatically collected multimodal annotations.

Who it’s for

This tool is primarily for fashion designers and researchers in computer vision who want to create realistic fashion illustrations and edit garment images using a variety of control inputs.

Highlights

• Multimodal Control: Supports a combination of text, body poses, and garment sketches to guide image generation.
• Latent Diffusion Base: Applies latent diffusion models to the fashion domain for improved realism and coherence.
• Detailed Datasets: Provides extended versions of the Dress Code and VITON-HD datasets with multimodal annotations.
• Pre-trained Models: Offers models and checkpoints available via torch.hub for easier implementation.