zsyOAOA/ResShift

What it solves

ResShift addresses the slow inference speed of diffusion-based image super-resolution (SR) models. Traditional diffusion models often require hundreds or thousands of sampling steps, and while acceleration techniques exist, they frequently result in over-blurry images. ResShift aims to provide high-quality super-resolution and image restoration without sacrificing performance for speed.

How it works

ResShift utilizes a novel diffusion model that constructs a Markov chain to transfer between low-resolution and high-resolution images by shifting the residual between them. This approach significantly improves transition efficiency, reducing the required sampling steps. It also employs a specialized noise schedule to control shifting speed and noise strength during the diffusion process, allowing it to achieve state-of-the-art results in as few as 15 sampling steps (and as few as 4 steps in later versions).

Who it’s for

This project is for researchers and developers working on image restoration, specifically those needing efficient, high-quality super-resolution, image deblurring, inpainting, and blind face restoration.

Highlights

• High Efficiency: Achieves superior or comparable performance to state-of-the-art methods with only 15 sampling steps.
• Versatile Applications: Supports real-world image super-resolution, bicubic super-resolution, image deblurring, natural image inpainting, face inpainting, and blind face restoration.
• Reduced Blur: Eliminates the need for post-acceleration techniques that typically lead to blurry results.
• Flexible Control: Uses an elaborate noise schedule to manage the diffusion process.