estebanpdl/osintgpt

What it solves

osintgpt helps analysts examine large collections of research documents without mixing different projects or losing track of the original source of information. It allows users to index their own documents and ask questions based specifically on that material, ensuring answers are grounded in the provided data with clear citations.

How it works

The tool creates isolated projects that index documents using a RAG (Retrieval-Augmented Generation) approach. It converts various file formats (PDF, Word, Excel, CSV, JSON, etc.) into a searchable format, creates embeddings for the text, and stores them in a vector database (SQLite, Qdrant, or Postgres). When a user asks a question, the system can use semantic similarity, exact text matches, or graph relationships to find relevant material and generate an answer using a configured LLM.

Who it’s for

It is designed for OSINT (Open Source Intelligence) analysts and researchers who need to query their own private document collections while maintaining strict data provenance and the option for fully local processing.

Highlights

• Multi-Provider Support: Compatible with OpenAI, Gemini, Anthropic, Voyage, and Ollama for generation and embeddings.
• Privacy Options: Supports a fully local pipeline using Ollama and sentence-transformers so no data leaves the machine.
• Extensive File Support: Includes 29 readable extensions, including support for scanned PDFs via vision-based transcription.
• Robust Indexing: Features checkpointing to resume interrupted indexing runs and built-in rate limiting/pacing for paid APIs.
• Graph Capabilities: Can export sourced graph relationships to CYPHERL for use in Neo4j or Memgraph.