stable-diffusion.cpp: a lightweight C++ inference engine for image and video diffusion models

What it solves

stable-diffusion.cpp provides a lightweight, high-performance way to run diffusion models for image and video generation without needing heavy external dependencies or complex Python environments. It allows users to run these models locally on a wide variety of hardware, including CPUs and various GPU backends.

How it works

Built on the ggml library, the project implements diffusion model inference in pure C/C++. It supports a vast array of model architectures (such as Stable Diffusion 1.x/2.x/3.x, FLUX, and Wan) and weight formats including GGUF, Safetensors, and PyTorch checkpoints. It utilizes various hardware acceleration backends like CUDA, Metal, Vulkan, and OpenCL to optimize performance across different platforms including Linux, Windows, macOS, and Android.

Who it’s for

Developers and users who want to run image and video generation models locally with minimal overhead, as well as those seeking a portable, C++ based backend for integrating diffusion models into other applications.

Highlights

• Broad Model Support: Supports a wide range of image generation, image editing, and video generation models (e.g., SDXL, FLUX.2, Wan2.1).
• Hardware Versatility: Runs on CPU (AVX/AVX2/AVX512), CUDA, Vulkan, Metal, OpenCL, and SYCL.
• Lightweight Implementation: Pure C/C++ implementation with no external dependencies, similar to llama.cpp.
• Advanced Features: Includes support for LoRA, ControlNet, Latent Consistency Models (LCM), and image upscaling via ESRGAN.
• Memory Optimization: Features Flash Attention and VAE tiling to reduce VRAM/RAM usage.
• Cross-Platform: Compatible with Linux, Windows, macOS, and Android.