bcmi/libcom

What it solves

Libcom is a comprehensive toolbox designed to make image composition (or object insertion) look realistic. It addresses the common inconsistencies in appearance, geometry, and semantics that occur when a foreground object is inserted into a background image.

How it works

The library integrates multiple specialized AI models and traditional methods to handle different aspects of the composition process. It provides tools for:

• Placement and Evaluation: Predicting the most rational location and scale for an object (FOPA) and scoring the rationality of placement (OPA).
• Visual Harmony: Adjusting foreground illumination (ImageHarmonizationModel) or artistic style (PainterlyHarmonizationModel) to match the background, and using color transfer for basic matching.
• Realism Enhancements: Generating plausible shadows (ShadowGenerationModel) and reflections (ReflectionGenerationModel) for the inserted object.
• Generative Composition: Using advanced models like OSInsert and KontextBlendingHarmonization to insert objects into bounding boxes while adjusting pose, view, or illumination.
• Quality Control: Localizing inharmonious regions (MadisNet) and evaluating overall harmony and compatibility (HarmonyScoreModel, FOSScoreModel).

Who it’s for

Developers and researchers working on computer vision, image editing, and synthetic data generation who need a unified interface for complex image composition tasks.

Highlights

• All-in-one interface: Solves various composition problems with a single import libcom command.
• Diverse task coverage: Covers blending, harmonization, shadow/reflection generation, and object placement.
• Generative AI integration: Includes both conservative and aggressive modes for adjusting object pose and view during insertion.
• Online Workbench: Provides a web-based interface for testing the toolbox's capabilities.