sshh12/llm_osint

LLM OSINT is a proof-of-concept method of using LLMs to gather information from the internet and then perform a task with this information.

What it solves

The project addresses the problem that LLMs, when given tools to search the web, often perform shallow research and return early with incomplete answers. It provides a method to generate thorough, multi-step open-source intelligence (OSINT) by having an LLM orchestrate multiple focused web searches.

How it works

It uses a two-level architecture: a "knowledge agent" that plans and accumulates information, and multiple "web agents" that perform actual searches. Initially, a web agent does a broad search. Then the knowledge agent analyzes the results and identifies "deep dive" areas. For each area, a new web agent is spawned to gather more focused information. This process repeats for several rounds, building a knowledge base. Finally, the accumulated knowledge is fed to the LLM to answer a question or perform a task.

Who it’s for

This is for developers and researchers interested in using LLMs for OSINT or automated research. The README includes examples like looking up a person, generating psychological reports, creating resumes, and even crafting phishing emails (for educational/security purposes). It could also be of interest to security professionals or anyone studying AI capabilities.

Highlights

  • Proof-of-concept, as seen in The Wall Street Journal, for LLM-driven OSINT.
  • Uses a knowledge agent to orchestrate iterative deep dives, avoiding shallow answers.
  • Demonstrates capabilities like person lookup, resume generation, and ad personalization.
  • Includes a privacy note urging ethical use and respect for individuals' privacy.

Related

  • Project
    sopaco/cortex-mem

    A Rust-based long-term memory framework for AI agents that uses a three-tier hierarchical architecture and a virtual filesystem to provide persistent, context-aware interactions.

  • Project
    simple-agent-lab/RSIHub

    RSIHub is a file-based framework that lets you improve an AI agent by repeatedly testing candidate variants against a fixed evaluator, keeping a full history of what changed and why, and only accepting changes that are verified to improve performance.

  • Project
    SakanaAI/mass

    MASS is a research framework that enables recursive self-improvement in AI models by alternating between optimizing multi-agent workflows and using the resulting data to fine-tune the model.

  • Project
    OTT-Cybersecurity-LLC/lyrie-ai

    An autonomous security agent that performs automated penetration testing, LLM red-teaming, and provides a cryptographic identity standard for AI agents via the Agent Trust Protocol.