Linfeng-Tang/SwinFusion

What it solves

SwinFusion is designed to solve the problem of general image fusion, where information from multiple source images (such as infrared and visible light, or different exposures) is combined into a single image that contains the most relevant information from all sources. It addresses the challenge of learning long-range dependencies across different domains of image data.

How it works

The project uses a Swin Transformer-based architecture to perform cross-domain long-range learning. This allows the model to capture global context and detailed features from various image types, which it then fuses into a high-quality result. The framework is designed to be general, meaning it can be applied to multiple types of fusion tasks without needing a completely different architecture for each.

Who it’s for

This tool is for researchers and developers working in computer vision and image processing, specifically those focusing on multi-modal image fusion (e.g., combining infrared and visible light images) and digital photography fusion (e.g., multi-exposure or multi-focus images).

Highlights

• General Purpose: Supports five different fusion scenarios: Visible and Infrared (VIF), Visible and Near-infrared (VIS-NIR), Medical (PET-MRI, CT-MRI), Medical, and Multi-Exposure (MEF) and Multi-Focus (MFF).
• Transformer-based: Leverages the Swin Transformer to handle long-range dependencies in image data.
• Comprehensive Testing: Provides training and testing scripts for a variety of standard datasets including MSRS, Nirscene, and Harvard medical datasets.
• Qualitative Superiority: Demonstrates improved results over state-of-the-art methods like U2Fusion and DenseFuse across multiple domains.