Joe-B-Security/awesome-prompt-injection

Overview

awesome-prompt-injection is a curated, community-maintained list of resources about prompt injection — a security vulnerability in AI systems built on large language models (LLMs). The README explains that attackers can embed hidden instructions in text the model reads, tricking it into doing unintended things. The project collects links to articles, tutorials, research papers, tools, and training challenges to help people understand, test, and mitigate this risk.

What it contains

The repository is organized into clear sections:

• Introduction – A plain-language explanation of prompt injection, including its direct and indirect forms, why it’s dangerous, and why there is no perfect defense.
• Introduction Resources – Key references like OWASP’s LLM Top 10 and Meta’s “Rule of Two” framework.
• Articles & Blog Posts – Deep dives and real-world examples from industry experts (e.g., Simon Willison, Microsoft, OpenAI, Snyk).
• Tutorials – Hands-on guides and videos for learning by doing.
• Research Papers – Academic papers covering attack techniques, defense strategies, and benchmarks (e.g., adaptive attacks, tool-hijacking).
• Tools – Open-source software for scanning, detecting, or containing prompt injection (e.g., Garak, OWASP Agent Memory Guard, Augustus).
• CTF – Free training platforms and capture-the-flag challenges like Gandalf and PromptTrace.
• Community & Contributing – Pointers for getting involved.

Purpose

It serves as a single entry point for anyone needing to understand, test, or defend against prompt injection. The README itself explains that the field is evolving, with no known universal fix, so the curated resources focus on practical approaches (like constraining what a compromised model can do) rather than perfect detection.

Target audience

The list is aimed at developers, security professionals, researchers, and AI enthusiasts who build or audit LLM-based applications. It’s useful both for learning the basics and for staying current with advanced techniques.

Notable highlights

• The README’s own introduction is a clear, non-technical explanation of why prompt injection is a serious security problem.
• It links to recent research showing that even published defenses can be bypassed by adaptive attacks — stressing the need for robust architectural safeguards.
• The resources cover both offensive and defensive perspectives, including tools for penetration testing and for containment.

In short, this project is a living bibliography that tracks the state of the art in one of the most pressing security challenges of AI-driven software.