Softlandia-Ltd/vision-is-all-you-need

What it solves

This project implements a Vision RAG (V-RAG) architecture that eliminates the need for traditional text chunking when processing documents. Instead of breaking text into pieces, it treats document pages as images to maintain visual structure and context.

How it works

1. Image Conversion: PDF pages are converted into images using pypdfium.
2. Visual Embedding: A Vision Language Model (VLM), specifically ColPali, is used to generate embeddings directly from these images.
3. Vector Storage: These embeddings are stored in a QDrant vector database.
4. Retrieval: When a user submits a query, the VLM generates a query embedding to find the most similar page images in the database.
5. Generation: The retrieved images and the original query are passed to a multimodal model (GPT-4o or GPT-4o-mini) to generate a final response.

Who it’s for

Developers and AI engineers looking to implement RAG systems that can handle documents without the complexity of text extraction and chunking.

Highlights

• Chunk-free indexing: Embeds pages as vectors directly from images.
• Multimodal Pipeline: Combines ColPali for embedding and GPT-4o for final answer generation.
• Integrated Stack: Includes a backend deployed via Modal and a companion frontend for interaction.