Oqura-ai/deepdoc

deepdoc – Local Deep‑Research Report Generator

What it does

• Takes a folder of local documents (PDF, DOCX, PPTX, images, TXT, Markdown) and builds a searchable vector index with Qdrant.
• Lets you specify a report topic and outline. A planner LLM creates a section structure that you can edit.
• For each section a research agent:
  1. Generates focused queries.
  2. Retrieves candidate chunks from Qdrant.
  3. Uses TypeSafe’s Jev model to score relevance, evidence quality, contradictions and possible prompt‑injection content.
  4. Filters/reranks the chunks, checks whether every subsection has enough evidence, and, if not, issues new queries (up to a configurable reflection limit).
  5. Passes the vetted evidence to a writing LLM, which composes the final markdown for that section.
• All sections are stitched together into a single markdown report saved in output_folder.

Key components

• Qdrant – local vector database for fast similarity search.
• Jev (TypeSafe) – a classifier that returns typed probabilities; it acts as a gate‑keeper to ensure only high‑quality evidence reaches the writer.
• LLM – default is OpenAI gpt‑4o‑mini, but the config is pluggable (e.g., Mistral, Anthropic).
• Python 3.10+ – orchestrates the pipeline; configuration lives in configuration.py.
• uv – lightweight tool for virtual‑env and dependency management.

Why it matters

• Keeps all data local, so proprietary documents never leave your machine.
• Adds a verification layer (Jev) on top of raw vector retrieval, reducing hallucinations and improving citation quality.
• Fully configurable: you can tune retrieval size, concurrency, Jev thresholds, and retry limits to match your hardware or API quotas.

Typical workflow

1. Install uv, clone the repo, create and activate a virtual environment.
2. Copy .env.example → .env and add your OpenAI and TypeSafe API keys.
3. Spin up Qdrant with Docker (docker‑compose up --build).
4. Run python main.py, answer the CLI prompts (topic, outline, document folder).
5. Review the generated outline, approve or edit, then let the agents research and write.
6. Retrieve the finished markdown report from output_folder.

Who should use it

• Researchers or analysts who need a quick, structured summary of a private document collection.
• Teams building internal knowledge bases that require citation‑aware generation.
• Anyone experimenting with retrieval‑augmented generation (RAG) pipelines that need an extra evidence‑filtering step.

License

• MIT License (see LICENSE).

Contributing

• Open to issues and pull requests; see the CONTRIBUTING guidelines in the repo.