wuyoscar/Internal-Safety-Collapse

We built an adversarial codespace setup. Place any AI agent into a normal workflow inside it, and the agent will fill in whatever is missing.

What it solves

It addresses the challenge of eliciting harmful content and artifacts from frontier Large Language Models (LLMs) that have strong safety guardrails. It provides a framework to bypass these refusals for red-teaming, evaluation, and mitigation research.

How it works

The project introduces the Task-Validator-Data (TVD) framework. Unlike traditional prompt attacks that target the user or tool channels, TVD targets the environment. It places the LLM inside a simulated coding project where it must complete a task (e.g., filling a test case) to satisfy a validator script.

Because the model perceives failures as programming errors rather than safety violations, it enters a self-loop of fixing the code. To pass the validator, the model is forced to generate the specific harmful content required by the task, leading to what the authors call Internal Safety Collapse (ISC).

Who it’s for

This tool is designed for AI safety researchers, red-teamers, and developers working on mitigation research, such as training safety guardrails and classifiers.

Highlights

  • TVD Framework: Uses a self-loop harness (Task, Validator, Data) to bypass safety refusals via environment-based triggers.
  • Broad Efficacy: Demonstrated to trigger ISC across numerous frontier models, including GPT-5 series, Claude 4.8 Opus, and Fable 5.
  • Agentic Generation: Includes a harness for AI agents to collect harmful data and sensitive artifacts at scale.
  • Research Datasets: Enabled the creation of the AgentHazard (harmful task trajectories) and HarmProfile (harmful distribution characterization) datasets.

Related

  • Project
    confident-ai/deepteam

    An open-source red teaming framework for LLM systems that simulates adversarial attacks to uncover vulnerabilities and provides guardrails for production security.

  • Project
    NVIDIA/garak

    garak is an open‑source CLI framework that automatically probes large language models for security‑relevant failures (hallucination, prompt‑injection, data leakage, toxicity, jailbreaks, etc.). It supports Hugging Face, OpenAI, Bedrock, NIM, Replicate, local GGUF, and generic REST models, and provides a large library of ready‑made probes and detectors. Results are shown as PASS/FAIL tables and saved as JSONL logs for deeper analysis.

  • Dispatch
    Red-Teaming Large Language Models

    Hugging Face outlines the critical role of red-teaming in identifying LLM vulnerabilities to prevent harmful outputs, emphasizing the need for collaborative, adaptive evaluation methods.

  • Project
    UKGovernmentBEIS/inspect_ai

    A framework created by the UK AI Security Institute for evaluating large language models, featuring over 200 pre-built evaluations and support for model-graded scoring.