RiccardoBiosas/awesome-MLSecOps

What it solves

MLSecOps (Machine Learning Security Operations) is the practice of weaving security into every stage of the machine learning lifecycle—from data collection and model development to deployment, monitoring, and incident response. This project addresses the lack of a single, organized entry point for finding open-source tools, research, and learning material on AI security. It helps practitioners discover resources for adversarial testing, privacy preservation, supply-chain protection, and securing LLM applications and AI agents.

How it works

This is a curated index (an "Awesome" list) that groups hundreds of open-source tools, frameworks, research repositories, and reference materials into clear categories. The main sections include:

• Open-source security tools (model and artifact security, adversarial ML testing, embodied AI/robotics security)
• LLM security and red teaming tools (prompt injection scanners, jailbreak detectors)
• AI agent security and rogue-agent defenses
• Privacy-preserving ML (differential privacy, encrypted computation)
• AI supply-chain security (model signing, SBOM generation)
• Threat modeling, attack vectors, and MLOps infrastructure vulnerabilities
• Educational resources: blogs, books, community resources, infographics

The list also promotes the maintainer's newsletter, "The MLSecOps Hacker," which publishes deeper articles on topics like model serialization attacks and agent security.

Who it’s for

Security engineers, ML practitioners, developers, AI red teams, and anyone responsible for securing machine learning systems. Whether you are evaluating model robustness, testing an LLM application, auditing data privacy, or hardening an AI agent's runtime, this catalog gives you a starting point. The maintainer also encourages contributions and community participation.

Highlights

• Comprehensive coverage: from model serialization attacks to embodied AI security, the catalog spans offensive and defensive tools.
• Notable entries include ModelScan (model serialization protection), Garak (LLM vulnerability scanner), Agent-Wiz (automated threat modeling for agent workflows), and Microsoft's Agent Governance Toolkit.
• Includes both well-established libraries (Adversarial Robustness Toolbox, Foolbox, CleverHans) and newer research prototypes (BadVLA, Provael for robotics).
• Resources are grouped for quick navigation, with explicit notes on whether a project is a proof-of-concept or production-ready.
• Actively maintained: the README shows recent newsletter articles and badges for maintenance, with a clear contribution guide.