AgentSafeLabs/safelabs-eval

safelabs‑eval – Red‑team evaluation framework for AI agents

What it is – safelabs‑eval is an open‑source Python package that lets you automatically test the safety of agent‑style LLM applications (e.g., LangChain agents, CrewAI crews, AutoGen bots, etc.). It ships a library of 300 curated adversarial prompts covering ten security categories (the “ASI01‑ASI10” taxonomy, inspired by OWASP). For each prompt the framework sends the request to your agent (via an HTTP endpoint or a direct Python callable) and then runs a set of pattern‑based detectors (pure‑Python regex/heuristic checks) on the response. The result is a structured report that classifies each interaction as VULNERABLE, FAIL, UNCERTAIN, or PASS and can be output as human‑readable text or JSON for CI pipelines.

Why it matters – Modern LLM agents are being deployed in production, but there is little tooling for systematic adversarial safety testing. safelabs‑eval fills that gap without requiring any changes to the agent code, extra LLM calls, or proprietary infrastructure. It gives developers a quick, repeatable way to surface prompt‑injection, data‑leakage, tool‑use safety, hallucination, and other agent‑specific risks.

Key features

• Framework‑agnostic: works with any HTTP‑exposed agent or with adapters for popular agent libraries (LangChain, CrewAI, AutoGen, LlamaIndex, OpenAI Agents SDK, Google ADK, Semantic Kernel). Only the core package is needed; adapters are optional extras.
• 300 adversarial prompts: 30 prompts per ASI category, each labeled with a difficulty tier (tier 1‑3) and metadata such as MITRE ATLAS technique IDs.
• Zero‑LLM scoring: detectors are pure‑Python pattern matchers, so evaluation adds no extra model cost.
• CLI & Python API: safelabs run … for quick command‑line testing; run_eval() for programmatic use in scripts or test suites.
• Structured output: JSON reports include per‑prompt verdicts, confidence scores, and remediation hints, making them suitable for automated CI/CD checks.
• Extensible architecture: detectors inherit from a BaseDetector class; new detectors can be added and registered with the Scorer.

How to use it

1. Install the package (Python 3.11+): pip install safelabs-eval.
2. CLI – point it at an agent endpoint:

   safelabs run --target http://localhost:8000/chat --category all --output json
   

3. Python – wrap any callable that accepts a string and returns a string:

   from safelabs import run_eval
   async def my_agent(prompt: str) -> str: …
   result = asyncio.run(run_eval(my_agent, categories=["ASI01", "ASI06"]))
   result.summary()
   

4. Inspect the generated report; any VULNERABLE or FAIL entries should be reviewed and mitigated.

Limitations – The detectors rely on regex/heuristic patterns, so they may miss subtle or novel attacks and can produce false positives. The prompt set is English‑only and single‑turn; multi‑turn dynamics are only simulated within a single prompt. Confidence scores are heuristic evidence levels, not calibrated probabilities.

Who maintains it – The project is developed by Safe Labs AI Inc. and is released under the Apache 2.0 license. It includes a DOI‑linked dataset (SafeAgent‑300) and a research track with several pre‑print papers.

---

TL;DR – safelabs‑eval is a Python‑only, framework‑agnostic red‑team testing suite for LLM agents, providing 300 adversarial prompts and pattern‑based detectors to surface security weaknesses without extra model cost.