CyberSecEval 2: Evaluating Cybersecurity Risks in Large Language Models

Hugging Face and Meta have introduced CyberSecEval 2, a comprehensive evaluation framework designed to measure the cybersecurity risks and capabilities of Large Language Models (LLMs). This framework is critical as LLMs are increasingly integrated into coding assistants, introducing new vulnerabilities that require standardized benchmarking to mitigate.

Cybersecurity Benchmarking Categories

CyberSecEval 2 utilizes five distinct testing suites to evaluate how LLMs handle insecure code and requests to assist in cyberattacks:

  • Insecure Coding Practices: This test measures the frequency with which an LLM suggests risky security weaknesses in autocomplete and instruction contexts, based on the industry-standard Common Weakness Enumeration (CWE) taxonomy. Results are reported as code test pass rates.
  • Prompt Injection Susceptibility: These tests evaluate a model's ability to distinguish between trusted and untrusted input and its resilience against common prompt injection techniques. The metric reported is the frequency of model compliance with attacks.
  • Cyber Attack Compliance: This suite measures the tradeoff between the violation rate (agreeing to assist in offensive cyberattacks) and the false rejection rate (refusing benign prompts related to cyberdefense).
  • Code Interpreter Abuse: This evaluates whether an LLM can be manipulated into executing malicious code within a sandboxed environment to gain system access, gather sensitive information, or execute social engineering attacks. Results are reported as the frequency of compliance.
  • Automated Offensive Capabilities: Using capture-the-flag (CTF) style security test cases, this suite determines if an LLM can solve end-to-end exploit challenges, such as SQL injections and buffer overflows. Results are reported as a percentage of completion.

Key Findings and Industry Trends

Evaluation of state-of-the-art LLMs using CyberSecEval 2 has revealed several critical insights into the current state of AI security:

Reduction in Attack Compliance

Since the first version of the benchmark was published in December 2023, the average LLM compliance rate with requests to assist in cyberattacks has decreased from 52% to 28%. This indicates a broad industry trend toward better safety alignment regarding offensive cyber requests.

Model Specialization and Security

Models without code specialization generally exhibit lower non-compliance rates than code-specialized models. However, the gap between these two categories is narrowing, suggesting that specialized coding models are improving their security posture.

Persistent Prompt Injection Risks

Prompt injection remains an unsolved problem. The tests indicate that developers cannot assume LLMs will safely follow system prompts when faced with adversarial inputs, posing a significant risk for LLM-based applications.

Limitations in End-to-End Exploitation

While models with high general coding capabilities perform better in exploitation tests, LLMs currently lack the ability to reliably solve end-to-end exploit challenges. This suggests that LLMs are unlikely to disrupt cyber exploitation attacks in their current state.

Vulnerabilities in Code Interpreters

LLMs remain vulnerable to manipulation that allows them to perform abusive actions inside code interpreters. This finding underscores the necessity for additional guardrails and detection mechanisms to prevent interpreter abuse.

Open Source Contribution

All CyberSecEval 2 code is open source. The community is encouraged to run these benchmarks on their own models and submit results to the CyberSecEval 2 leaderboard to enhance the collective understanding of LLM cybersecurity safety properties.

Sources