bcmi/libcom

libcom – A One‑Stop Toolbox for Image Composition

What it does

• Provides a Python library (import libcom) that bundles a collection of ready‑to‑use models and utilities for image composition – the task of inserting a foreground object into a background image and making the result look realistic.
• Covers the whole pipeline:
  1. Placement – OPAScoreModel / FOPAHeatMapModel assess where an object should go and suggest the best location/scale.
  2. Appearance matching – colour transfer, image‑harmonisation (PCTNet, LBM), painterly harmonisation (PHDNet, PHDiffusion).
  3. Geometry & semantics check – FOSScoreModel evaluates geometric and semantic compatibility; InharmoniousLocalizationModel spots mismatched regions.
  4. Shadow & reflection synthesis – ShadowGenerationModel (GPSDiffusion) and ReflectionGenerationModel (RGDiffusion).
  5. Generative composition – KontextBlendingHarmonizationModel (Flux‑based) and OSInsertModel can actually generate the foreground‑in‑background image, optionally adjusting pose, lighting, or style.
  6. Simple baseline – get_composite_image does naïve copy‑paste with alpha or Poisson blending.

How you use it

import libcom
# naive blend
result = libcom.get_composite_image(fg, bg, mask)
# evaluate placement
score = libcom.OPAScoreModel(bg, fg).predict()
# harmonise illumination
harmonised = libcom.ImageHarmonizationModel(bg, fg).run()
# generate shadow
shadowed = libcom.ShadowGenerationModel(bg, fg).run()

All models are thin wrappers around published research code (e.g., PCTNet, LBM, PHDNet, GPSDiffusion). The library handles downloading the pretrained weights, preprocessing, and returning a Pillow/NumPy image.

Why it matters
Image composition is a core capability for photo editing, AR/VR content creation, synthetic data generation, and visual effects. Historically, each sub‑task required stitching together separate research implementations. libcom unifies them under a consistent API, making experimentation and downstream product integration much faster.

Installation & platform

• Requires Linux, Python 3.10+, PyTorch ≥ 2.6.
• Install via PyPI: pip install libcom (the package pulls the heavy model files on first use).
• Detailed docs, tutorials, and an online demo workbench are provided (https://libcom.ustcnewly.com/).

Community & future

• Open‑source under Apache 2.0, maintained by the Brain‑like Computing and Machine Intelligence Lab (SJTU).
• The authors invite contributions of new models and collaborations on inference acceleration.

---

TL;DR: libcom is a Python toolbox that bundles state‑of‑the‑art models for every step of realistic object insertion—placement scoring, colour/illumination harmonisation, shadow & reflection synthesis, and even generative composition—so you can go from a raw foreground‑background pair to a polished composite with a few function calls.