EasyJailbreak/EasyJailbreak

EasyJailbreak – A Python framework for LLM jailbreak research

What it is – EasyJailbreak is an open‑source Python library that lets researchers build, run, and evaluate jailbreak attacks on large language models (LLMs). It breaks the typical jailbreak workflow into reusable components (seed generation, selector, mutator, constraint, attacker, evaluator) and ships a collection of ready‑made “recipes” that re‑implement published attacks such as ReNeLLM, GPTFuzz, AutoDAN, PAIR, etc.

Why it matters – As LLMs are deployed in more products, understanding how they can be coaxed into disallowed behaviour (e.g., giving harmful instructions) is crucial for safety. EasyJailbreak provides a unified playground so that security researchers can:

• Compare many attack strategies on the same models and datasets.
• Plug in new mutation or evaluation methods without rewriting the whole pipeline.
• Generate reproducible reports of jailbreak prompts, model responses, and success scores.

Key concepts

Stage	Role
Seed	Initial malicious queries or prompts to start the attack.
Selector	Chooses the most promising seeds for the next round (e.g., random, MCTS, score‑based).
Mutator	Transforms a seed (re‑phrase, translate, add noise, cipher, etc.).
Constraint	Filters mutated prompts (e.g., removes harmless or overly long prompts).
Evaluator	Scores how successful a jailbreak is (generative judge, classification judge, pattern match, etc.).
Loop	Selector → Mutator → Constraint → Attack (run on target model) → Evaluator → back to Selector.

Built‑in attack recipes – The library ships implementations of 11 published jailbreak pipelines, each wiring specific selectors, mutators, constraints, and evaluators. For example:

• ReNeLLM – mutators like ChangeStyle and Rephrase with a generative‑judge evaluator.
• GPTFuzz – uses MCTS or UCB selectors and a classification‑judge evaluator.
• AutoDAN, PAIR, JailBroken, Cipher, DeepInception, MultiLingual, GCG, TAP, CodeChameleon – each with their own set of mutation rules.

Installation

# Only need the ready‑made attacks
pip install easyjailbreak

# For development / adding new components
git clone https://github.com/EasyJailbreak/EasyJailbreak.git
cd EasyJailbreak
pip install -e .

Requires Python ≥ 3.9.

Quick start (using a recipe)

from easyjailbreak.attacker.PAIR_chao_2023 import PAIR
from easyjailbreak.datasets import JailbreakDataset
from easyjailbreak.models.huggingface_model import from_pretrained
from easyjailbreak.models.openai_model import OpenaiModel

# Load models
attack_model = from_pretrained('lmsys/vicuna-13b-v1.5', model_name='vicuna_v1.1')
target_model = OpenaiModel('gpt-4', api_keys='YOUR_KEY')
 eval_model   = OpenaiModel('gpt-4', api_keys='YOUR_KEY')

# Load a benchmark dataset
dataset = JailbreakDataset('AdvBench')

# Instantiate the PAIR attack pipeline
attacker = PAIR(attack_model, target_model, eval_model, dataset)

# Run the attack and save results
attacker.attack(save_path='vicuna_gpt4_result.jsonl')

The call handles the whole selector‑mutator‑evaluator loop and writes a JSON‑lines report containing prompts, model replies, and scores.

DIY – building your own attacker

1. Load any model (local HuggingFace checkpoint, OpenAI‑compatible API, MiniMax, etc.) with a one‑line helper.
2. Create a seed via SeedRandom() or supply your own prompts.
3. Pick components – import a selector (RandomSelectPolicy), a mutator (Translate, ChangeStyle, etc.), optional constraints, and an evaluator (Evaluator_GenerativeJudge).
4. Wire them together in a custom class or script and run the loop.

Documentation & resources

• Paper: EasyJailbreak: A Unified Framework for Jailbreaking Large Language Models (arXiv 2403.12171) – describes design and experimental results.
• Website: http://easyjailbreak.org/ – interactive view of jailbreak results across models.
• Read‑the‑Docs: https://easyjailbreak.github.io/EasyJailbreakDoc.github.io/ – full API reference.
• Datasets: Hosted on HuggingFace under Lemhf14/EasyJailbreak_Datasets.
• Experimental data: Results of 11 recipes on 10 LLMs are downloadable via the provided Google‑Drive link.

Citation

@misc{zhou2024easyjailbreak,
  title={EasyJailbreak: A Unified Framework for Jailbreaking Large Language Models},
  author={Weikang Zhou and Xiao Wang and Limao Xiong and ... and Xuanjing Huang},
  year={2024},
  eprint={2403.12171},
  archivePrefix={arXiv},
  primaryClass={cs.CL}
}

---

EasyJailbreak is therefore a genuine, research‑grade tool for LLM security, not a simple tutorial or list. It gives practitioners a modular, reproducible way to explore how LLMs can be prompted into unsafe behaviour and how to measure such vulnerabilities.