zhengxuJosh/Awesome-RAG-Vision

Awesome-RAG-Vision: a curated list of advanced retrieval augmented generation (RAG) for Computer Vision

🎯 What is Awesome‑RAG‑Vision?

Awesome‑RAG‑Vision is a curated “awesome‑list” that gathers research papers, surveys, benchmarks and related resources about Retrieval‑Augmented Generation (RAG) for computer‑vision tasks. It is not a code library or a runnable system; rather, it organises the rapidly growing literature on how visual models (image, video, 3‑D, medical, embodied AI, etc.) can be combined with external retrieval mechanisms (databases, knowledge graphs, long‑form documents, memory banks) to improve understanding or generation.

📚 What you’ll find inside

Section	What it contains
Introduction	A short explanation of RAG in vision – why retrieval helps image captioning, VQA, video QA, generative models, etc.
Scope & Inclusion	Rules the maintainers use to decide which works count as visual RAG (e.g., must involve a retriever that supplies evidence to a generative model).
Resources	Links to workshops, tutorials and blog posts that teach how to build multimodal RAG pipelines (LangChain, LlamaIndex, LanceDB, Stable Diffusion, etc.).
Papers – Survey & Benchmark	A table of recent surveys (2023‑2026) that cover RAG in NLP/LLMs and the emerging multimodal RAG literature.
RAG for Vision	The core of the list – dozens of papers grouped by task:
• Visual Understanding – image captioning, VQA, open‑vocabulary detection, long‑tail recognition, etc.
• Long‑Video Understanding – scene‑level retrieval, streaming RAG, narrative video generation, embodied assistants.
• Visual Spatial Understanding – spatial description with hallucination correction.
• Multi‑modal – document‑centric RAG, knowledge‑graph‑augmented VQA, enterprise/industrial use‑cases.
• Visual Generation – image/video generation, 3‑D generation (not fully listed in the excerpt).
• Embodied AI – agents that retrieve visual knowledge while interacting with environments.
Related Components	Mentions retrievers, knowledge bases, evaluation suites that support RAG but are not end‑to‑end systems.
Citation	A ready‑to‑copy BibTeX entry for citing the list itself.

🛠️ How to use it

Explore a task – Click the relevant sub‑section (e.g., Image Understanding). Each entry shows title, authors, venue and a direct link to the PDF or arXiv page.
Find implementation clues – Many paper titles include “framework”, “plug‑and‑play”, or “benchmark”, which often point to accompanying code repositories. Follow the links to locate the code.
Learn the basics – The Resources section aggregates tutorials (LangChain, LlamaIndex, Stable Diffusion, etc.) that walk through building a multimodal RAG pipeline from scratch.
Stay up‑to‑date – The list is open‑source; you can submit a PR to add new RAG‑vision papers you discover.

📌 Who will benefit?

Researchers looking for a quick overview of the state‑of‑the‑art in vision‑centric RAG.
Engineers who want to locate recent papers that include code or datasets for building retrieval‑augmented vision systems.
Students preparing literature reviews or project proposals on multimodal AI.
Product teams scouting academic ideas for next‑generation visual assistants, document‑centric AI, or video‑analysis tools.

🔗 Quick links

Survey paper: Retrieval Augmented Generation and Understanding in Vision: A Survey and New Outlook (arXiv:2503.18016)
GitHub repo: https://github.com/zhengxuJosh/Awesome-RAG-Vision
License: MIT (free to reuse and remix).

This explanation is based solely on the repository’s README; the list itself does not contain executable software.