cure-lab/MMA-Diffusion

[CVPR2024] MMA-Diffusion: MultiModal Attack on Diffusion Models

What it solves

MMA-Diffusion addresses the security vulnerabilities of Text-to-Image (T2I) models, specifically their susceptibility to generating Not-Safe-For-Work (NSFW) content despite having safety mechanisms like prompt filters and post-hoc safety checkers. It demonstrates how these defenses can be bypassed to perform unauthorized editing of real individuals' imagery with NSFW content.

How it works

The framework employs a multi-modal attack strategy that targets both text and image modalities:

  • Text-modality attack: It generates adversarial prompts that can circumvent prompt filters to trigger the generation of prohibited content.
  • Image-modality attack: It uses adversarial images to bypass post-hoc safety checkers that are designed to prevent the synthesis of explicit imagery.

Who it’s for

This tool is primarily for researchers and developers working on AI security, specifically those focused on evaluating the robustness of T2I models and developing more effective defense mechanisms against adversarial attacks.

Highlights

  • Multi-modal approach: Combines textual and visual attacks to bypass multiple layers of safety guards.
  • Benchmark datasets: Provides a benchmark of 1,000 successful adversarial prompts and a comprehensive dataset of adversarial images for evaluation.
  • Realistic threat modeling: Focuses on the vulnerability of T2I models to the unauthorized editing of real people's images into NSFW content.