liu00222/Open-Prompt-Injection

What it solves

It provides a standardized toolkit for implementing, evaluating, and extending prompt injection attacks and defenses in LLM-integrated applications and agents, helping researchers and developers benchmark how these vulnerabilities affect model behavior.

How it works

The toolkit uses a factory-based approach to create models, tasks, and attackers. It allows users to define a target task (e.g., sentiment analysis) and an injected task (e.g., spam detection) to simulate attacks. For defense, it includes specialized components: DataSentinel for detecting if a prompt has been contaminated and PromptLocate for localizing the injected portion of the prompt to recover the original data.

Who it’s for

Security researchers, AI developers, and LLM practitioners who need to test the robustness of their AI agents and applications against prompt injection attacks.

Highlights

• Attack Simulation: Supports combined attack strategies to evaluate Attack Success Value (ASV).
• Detection Pipeline: Features DataSentinel for game-theoretic detection of prompt injections.
• Localization: Includes PromptLocate to identify and isolate injected prompts for data recovery.
• Extensible Framework: Uses configuration files to easily swap models (e.g., PaLM2, Llama, GPT) and tasks.