Skill Scanner – A Security Scanner for AI Agent Skills

What it is – An open‑source command‑line tool (and Python SDK) that statically analyses AI‑agent skill packages (e.g. OpenAI Codex, Cursor, Claude) to look for risky patterns such as prompt‑injection, data‑exfiltration, or malicious code. It combines several detection engines (YAML/YARA signatures, AST‑based data‑flow analysis, optional LLM‑as‑a‑judge, and a CEL‑based decision layer) and can be plugged into CI/CD pipelines.

---

Core capabilities

Capability	How it works	What it looks for
Static signature scanning	YAML rule files + YARA‑X patterns	Known malicious strings, unsafe commands, suspicious API calls
Byte‑code integrity	Checks compiled .pyc files	Tampered or injected byte‑code
Pipeline taint analysis	Parses shell pipelines for source‑sink flows	Command‑injection vectors
Behavioral / AST analysis	Builds an abstract syntax tree and follows data‑flow	Prompt‑injection, unsafe data handling
LLM semantic analysis (optional)	Sends the skill description and code to an LLM (Anthropic or OpenAI) and interprets the response as a finding	Context‑aware malicious intent that signatures miss
Meta‑analyzer (optional)	Correlates findings, filters false‑positives, ranks severity	Reduces noise from overlapping rules
Cloud‑based AI Defense	Calls Cisco’s AI‑Defense service for extra screening	Cloud‑hosted threat intelligence
VirusTotal integration	Hash lookup / optional file upload	Known malware binaries bundled with a skill
CEL decision layer	Uses the official cel-go runtime to evaluate typed facts and enforce policy (off / shadow / enforce)	Deterministic, bounded rule‑application

The scanner can be run on a single skill directory or recursively on a repository of skills, and it can output results in many formats (JSON, SARIF, HTML, Markdown, etc.).

---

Typical workflow

1. Install – pip install cisco-ai-skill-scanner (or via uv).
2. Configure – set environment variables for any LLM or cloud APIs you want to use.
3. Run –

   skill‑scanner scan /path/to/skill \
       --use-behavioral --use-llm --enable-meta \
       --format sarif --output results.sarif
   

   The command runs the static, behavioral, and optional LLM engines, applies the CEL policy, and writes a SARIF report that GitHub Code Scanning can display.
4. CI/CD – add the reusable GitHub‑Actions workflow or a pre‑commit hook so every pull request is automatically scanned.
5. Review – examine the HTML or SARIF report; findings are classified by severity (info, low, medium, high, critical). A “no findings” result is best‑effort only – it does not guarantee the skill is safe.

---

Who should use it

• Developers of AI agents who ship reusable skill packages and need an automated guard‑rail before publishing.
• Security teams that want to integrate AI‑skill scanning into existing CI pipelines.
• Platform operators (e.g., internal AI marketplaces) that must enforce a baseline security policy on contributed skills.

---

Limitations (as stated in the README)

• The scanner is best‑effort; it cannot detect novel or zero‑day attack techniques.
• False positives and false negatives are expected; policy tuning and manual review are recommended.
• Certain advanced analyses (LLM, cloud AI‑Defense) require external API keys and may incur cost.
• The CEL rule set is currently in shadow mode, meaning findings are reported but not automatically suppressed.

---

Quick‑start cheat sheet

# Install
pip install cisco-ai-skill-scanner

# Simple scan (static only)
skill-scanner scan ./my-skill

# Full scan with behavioural + LLM + meta filtering
skill-scanner scan ./my-skill \
    --use-behavioral --use-llm --enable-meta \
    --policy strict --format html --output report.html

# CI integration (GitHub Actions)
# Add the reusable workflow shown in docs/github-actions.md

---

Bottom line – Skill Scanner is a genuine, actively maintained security‑analysis tool aimed at the emerging domain of AI‑agent skill packages. It blends traditional static‑analysis techniques with optional LLM‑driven semantic checks and a programmable CEL policy layer, making it suitable for both local developer use and automated CI/CD enforcement.