TencentARC/ColorFlow

What it solves

ColorFlow addresses the difficulty of automatically colorizing black-and-white image sequences (such as comics or cartoons) while keeping the colors of characters and objects consistent across different frames. It eliminates the need for per-character fine-tuning or manual identity embedding extraction, which are common hurdles in industrial applications.

How it works

ColorFlow uses a three-stage diffusion-based framework:

1. Retrieval-Augmented Pipeline (RAP): Finds and retrieves relevant color references to guide the colorization process.
2. In-context Colorization Pipeline (ICP): Uses a dual-branch design (one for color identity extraction and one for colorization) and self-attention mechanisms to match colors from the references to the target image.
3. Guided Super-Resolution Pipeline (GSRP): Ensures the final output is high-quality and high-resolution.

Who it’s for

This tool is designed for artists and industrial creators working with sequential art, such as manga, comic book artists, and animation studios who need to maintain strict color identity across multiple images.

Highlights

• Retrieval-Augmented Approach: Uses external color references instead of requiring model retraining for new characters.
• Identity Consistency: Specifically designed to preserve the identity of characters and objects across a sequence.
• Diffusion-Based: Leverages the power of diffusion models for high-quality visual generation.
• Comprehensive Evaluation: Includes ColorFlow-Bench, a new benchmark for reference-based colorization.