Hugging Face Open Preference Dataset for Text-to-Image Generation

The Data is Better Together community has released an Apache 2.0 licensed open preference dataset for text-to-image generation. This dataset provides preference pairs across various image generation categories and prompt complexities to enable better alignment and fine-tuning of open-source text-to-image models.

Dataset Composition and Input Pipeline

The open-image-preferences-v1 dataset was constructed using a pipeline that combined real-world prompts, synthetic enhancement, and multi-model image generation.

Input Prompt Sourcing and Cleaning

Prompts were sourced from Imgsys, a generative image model arena hosted by fal.ai. Because these prompts reflect real-life usage, they required cleaning to remove duplicates and toxic content. To ensure the dataset remained safe, the team implemented a multi-model filtering approach using two text-based and two image-based classifiers, followed by a manual review of every image by the Argilla team.

Synthetic Prompt Enhancement

To increase data diversity and quality, the team used a distilabel pipeline to synthetically rewrite prompts into different levels of complexity and style. This process created three tiers of prompts:

  • Default: Basic prompts (e.g., "a harp without any strings").
  • Stylized: Prompts enhanced with style and detail (e.g., adding anime style and pastel backgrounds).
  • Quality: High-detail prompts focusing on lighting, resolution, and texture (e.g., adding "photorealistic" and "soft golden hour light").

Prompt Categories and Complexities

Prompts were organized into 11 main categories based on sources like google/sdxl and Microsoft's AI art prompting guide. These include Cinematic, Photographic, Anime, Manga, Digital art, Pixel art, Fantasy art, Neonpunk, 3D Model, Painting, Animation, and Illustration. The dataset also includes simplified and complex versions of the same prompt to test model performance across varying levels of detail.

Image Generation and Model Comparison

Two high-performing models from different model families were used to generate images for the preference pairs: stabilityai/stable-diffusion-3.5-large (SD3.5-XL) and black-forest-labs/FLUX.1-dev (FLUX-dev).

Model Performance by Category

Analysis of the annotated data revealed that model strengths vary by category:

  • FLUX-dev: Performed better in 3D Model, Anime, and Manga categories.
  • SD3.5-XL: Performed better in Cinematic, Digital art, Fantasy art, Illustration, Neonpunk, Painting, and Pixel art.
  • Ties: Photographic and Animation categories resulted in ties.

Annotator Alignment

Using the Hugging Face datasets SQL console, the team found that SD3.5-XL was slightly more likely to win overall in the test setup, with annotator agreement remaining balanced—neither too high (indicating a task too easy) nor too low (indicating a task too hard).

Model Fine-Tuning and Validation

To validate the dataset's utility, the team performed a LoRA fine-tune of the FLUX.1-dev model. By using the preferred samples as expected completions and omitting rejected samples, the fine-tuned model showed significant improvement in art and cinematic scenarios where the original FLUX-dev model had previously been lacking.

Community Contribution and Scale

Over 250 community members contributed to the project in less than two weeks, annotating 10,000 preference pairs. With an annotator overlap of 2/3, this resulted in over 30,000 total responses.

Available Resources

Sources