praetorian-inc/augustus

What it solves

Augustus is a vulnerability scanner designed to test the security and robustness of Large Language Models (LLMs). It helps security professionals identify weaknesses such as prompt injection, jailbreaks, and data extraction vulnerabilities that could be exploited by adversarial actors.

How it works

Augustus uses a pipeline architecture to execute adversarial attacks. It selects a probe (an attack pattern), optionally applies a buff transformation (like Base64 encoding or translation) to evade detection, and sends the prompt to the target LLM via a generator (provider integration). The response is then analyzed by a detector (using pattern matching or an LLM-as-a-judge) to determine if the attack succeeded.

For complex attacks, it employs specialized engines:

• Single-turn engine: Refines prompts iteratively for probes like PAIR or TAP.
• Multi-turn engine: Maintains a conversation history to bypass safety filters using strategies like Crescendo (gradual escalation), GOAT (adaptive technique switching), Hydra (backtracking on refusal), and Mischievous User (casual persona probing).

Who it’s for

It is built for security professionals and enterprise teams who need a production-ready tool for LLM red-teaming and vulnerability assessment, rather than a research-oriented framework.

Highlights

• Extensive Attack Library: Over 210 vulnerability probes across 47 categories, including jailbreaks, RAG poisoning, and agent attacks.
• Broad Provider Support: Integrates with 28 LLM providers, including OpenAI, Anthropic, Azure, Bedrock, Vertex AI, and local options like Ollama.
• Comprehensive Detection: Over 90 detectors, including LLM-as-a-judge and the Perspective API.
• Production-Ready Features: Written in Go for high performance, featuring concurrent scanning, rate limiting, and a single-binary distribution.
• Flexible Output: Generates reports in Table, JSON, JSONL, and HTML formats.