mll-lab-nu/Awesome-Spatial-Intelligence-in-VLM

A paper list for spatial reasoning

What is this project?

Awesome Spatial Intelligence in VLMs is a curated, community-maintained list of research resources about spatial intelligence in Vision-Language Models (VLMs) — AI models that process both images and text. The list collects papers, datasets, and benchmarks that help researchers and engineers track progress in teaching these models to understand space: where objects are, how they relate in 3D, depth, layout, and similar geometric concepts.

What does it contain?

The README presents a large table of methods — research papers — each with:

  • A title and link to the paper (arXiv)
  • A visual introduction image
  • A publication date
  • A link to code (GitHub or project page) when available

These methods are split into two categories:

  • Visual-based methods — approaches that use images, videos, or 3D data as input
  • Text-based methods — approaches that work primarily with text input (the README shows this section header, though the table itself is truncated in the provided excerpt)

The table of contents also lists Datasets & Benchmarks and Findings & Applications sections, though the README excerpt doesn't show their contents.

How is it organized?

It follows the standard "awesome list" format: a single README with a table of contents, tables of resources, and badges (e.g., the awesome.re badge, GitHub star counts, conference publication badges like NeurIPS, CVPR, ICCV). Entries are sorted roughly by date, newest first.

Who is it for?

Researchers and engineers working on multimodal AI who want a quick index of the latest work on spatial reasoning in VLMs — what methods exist, which datasets and benchmarks are used, and where to find code.

How can you contribute?

The README explicitly welcomes contributions via Pull Request — anyone can add excellent resources they find.

Key facts

  • Type: Curated resource list (not a software library or model)
  • Focus: Spatial intelligence in Vision-Language Models
  • Content: Methods, datasets, benchmarks, findings, applications
  • Maintenance: Community-contributed via PRs
  • Coverage: Papers from roughly 2023–2026, with many from 2025

関連

  • プロジェクト
    OpenMOSS/MOSS-VL

    MOSS‑VL は、リアルタイムライブ動画ストリーム（MOSS‑VL‑Realtime）での対話と、オフライン高品質動画QA（MOSS‑VL‑Instruct）およびさらなる学習用のベースチェックポイントを備えたオープンウェイト、110億パラメータの動画言語モデルシリーズです。クロスアテンションアーキテクチャにタイムスタンプと3次元回転位置埋め込み（XRoPE）を採用し、中断可能なストリーミング、能動的沈黙、動的修正を実現。24GB GPU一台で動作可能な量子化チェックポイントも提供。リポジトリには即時実行可能な推論スクリプト、FastAPI/WebSocketサーバー、SGLangベースのサービングバックエンド、ファインチューニングスクリプト（パラメータ全体およびLoRA）、および広範なベンチマーク結果が含まれます。

  • プロジェクト
    InternLM/Intern-Decision

    Intern-Decision は、状態と画像を入力として、複数選択、スコアリング、yes/no 質問など、さまざまな形式の質問に対して補正された確率付きの意思決定を生成する、Qwen3.5 をベースにファインチューニングされたモデルです。

  • プロジェクト
    sign/translate

    手話動画を話す言語のテキストと音声に、また話す言語をアバターまたはGANを用いた手話にリアルタイムで双方向に変換するシステム。

  • プロジェクト
    zhengli97/Awesome-Prompt-Adapter-Learning-for-VLMs-CLIP

    A curated list of papers, datasets, and methods for prompt/adapter learning in vision-language models (like CLIP), with comparison tables and links to code.