BBC-Esq/VectorDB-Plugin

Program that lets you ask questions about your documents, audio, and video files.

What it solves

This project provides a way to create a local vector database from a wide variety of file types, allowing users to chat with their own documents, images, and audio files to get more reliable LLM responses through Retrieval Augmented Generation (RAG).

How it works

The system processes inputs through a pipeline:

  1. Ingestion: It accepts a broad range of formats including documents (.pdf, .docx, .txt, .html, .md, .csv, .xlsx, .rtf, .eml, .msg), images (.png, .jpg, .jpeg, .bmp, .gif, .tif, .tiff), and audio files (.mp3, .wav, .m4a, .ogg, .wma, .flac).
  2. Processing: It extracts text from documents, generates descriptions for images, and transcribes speech from audio files.
  3. Storage: The processed content is embedded and saved into a vector database for efficient searching.
  4. Retrieval and Generation: When a user asks a question (via text or voice), the system retrieves relevant chunks of information from the vector database and sends them to an LLM (such as a local model, Kobold, LM Studio, or ChatGPT) to generate a grounded answer.

Who it’s for

Windows users who want to build a local knowledge base from multimodal data (text, images, audio) and use it to augment their LLM queries.

Highlights

  • Multimodal Ingestion: Supports a vast array of document, image, and audio formats.
  • Flexible LLM Integration: Works with local models, Kobold, LM Studio, and ChatGPT.
  • Voice Interaction: Supports both voice-recorded questions and text-to-speech for responses.
  • GUI-based: Includes a graphical user interface for easier management.