ChocoWu/Awesome-Scene-Graph-Generation

What is this project?

A curated "awesome list" — a single, well-organized repository where the authors collect key resources about scene graph generation (SGG). It is a reference index, not a software package itself, but it links out to many tools, papers, and datasets.

What is a scene graph? (from the README's intro)

A way to describe a scene (from text, image, or video) as a graph:

• Nodes = objects, with category labels, attributes, and bounding boxes (where the object is)
• Edges = pairwise relationships between objects

Think of it as turning a picture into a structured description like "person — sits on — chair."

What's inside

• 🏷️ Datasets: A large table of ~30+ datasets covering many modalities — images, video, 3D, 4D, text, satellite imagery, traffic scenes, egocentric video, and more. Each row lists stats like object classes, relationship classes, and instance counts.
• 🧩 Scene Representation: Papers organized into 8 ways to represent a scene — patches/tokens, object-centric slots, neural implicit fields (like NeRF), explicit 3D, dynamic/4D/video, world-model latent, language-grounded, and symbolic/program-based.
• 🍕 Scene Graph Generation: Papers split by task type — 2D image SGG, panoptic SGG, spatio-temporal video SGG, audio SGG, 3D/4D SGG, textual SGG, map-space SGG, universal SGG, and world SGG.
• 🥝 Applications: How scene graphs are used — image/video retrieval, captioning, generation, visual reasoning, 3D scene generation, reducing hallucination in vision-language models, and more.
• 🤶 Extras: Evaluation metrics, toolkits, workshops, surveys, and notable works.

Toolkit highlights (linked in the README)

Several tools are listed:

• Stanford Scene Graph Parser, SceneGraphParser, FactualSceneGraph — text-to-scene-graph parsers
• Scene-Graph-Benchmark.pytorch, SGG-Benchmark — benchmark codebases (SGG-Benchmark is described as an "up-to-date and easy-to-run" implementation of common approaches)
• SGG-Annotate — a modern annotation tool for visual relationships in COCO format
• RelateAnything — real-time, open-vocabulary relation prediction from any inputs

News & recent additions by the maintainers

• OmniSG — a unified toolkit for text, image, video, and 3D scene graph parsing, with a web UI demo and evaluation scripts
• VSG-Annotator — a tool for annotating video scene graphs
• Scene Graph Series — a project website collecting all their scene-graph-related work
• WACV26 Workshop on "Scene Graph for Structured Intelligence" (a computer-vision conference workshop)

Community & discussions

The repo runs a "Hot Topics" section in the Issues tab — open threads like:

• "Are scene graphs still a good way to represent and understand scenes?"
• "Do World Models need Objects?"

The maintainers explicitly invite contributions, collaboration, and discussion about the future of the field.