DeepSeek v4.1 Flash Hacking Performance Analysis

DeepSeek v4.1 Flash achieved a perfect score on the Enclave AI hacking benchmark, successfully gaining code execution on all 11 vulnerable targets while failing to compromise any of the four fixed controls. The model demonstrated high cost-efficiency, with accepted runs costing only $4.65 due to heavy reliance on input caching.

High-Efficiency Attack Execution

DeepSeek v4.1 Flash completed the benchmark using 2,349 Bash commands over approximately 2 hours and 38 minutes of active model time. The median successful run took 4 minutes and 38 seconds.

Cost efficiency was driven by caching: of the 268.3 million input tokens processed, 266.2 million were cached, reducing the total cost of accepted runs to $4.65 (and $5.14 including failed attempts).

Analysis of Attack Vectors

An audit of the attack paths revealed that DeepSeek v4.1 Flash utilized both intended vulnerabilities and alternative routes available in the test environment.

Jenkins: Complex Logic and Timing Attacks

DeepSeek successfully executed the planned attacks on Jenkins targets. In one instance, it bypassed security boundaries by creating a file that pointed Jenkins toward a second file, allowing it to read private controller credentials and execute code via the built-in script tool. In another challenge involving a timing problem during file uploads, the model successfully paused an upload after one byte, changed the destination, and finished the upload to write a script into a protected location.

Nextcloud: Access Control Bypass

DeepSeek demonstrated strong source-code analysis by identifying an error in how Nextcloud remembered access decisions. The model created an approved access result for a read request and then reused that result for a write request to replace a PHP template, leading to code execution.

Grafana: Unplanned Route Discovery

In the Grafana challenges, DeepSeek consistently found a shorter, unplanned route. Instead of following the intended file-path handling issue, it placed executable files in a temporary plugin folder and requested Grafana to load that folder as a plugin. This allowed the model to gain execution in under 90 seconds across all three runs.

Benchmark Implications and Model Behavior

DeepSeek v4.1 Flash's performance highlights a critical distinction between outcome-based scoring and path-level auditing. While the model scored 11/11, only six runs followed the planned attack paths. Five runs utilized alternative routes specific to the private benchmark environment.

This behavior indicates that a hacking agent prioritizes the fastest working route over the intended path. Consequently, Enclave AI has updated the benchmark to close these extra routes and implement stricter checks on attack paths to ensure models are tested against specific vulnerabilities.

Community and Technical Perspectives

Technical discussions surrounding the model's performance provide a nuanced view of its agentic capabilities compared to other frontier models.

"Because it's a heavy reasoner, it sits near Gemini 3.7 Flash on the Pareto front... Closer than expected to the top open weights models (GLM 5.3 and Kimi K3) in agentic coding, at lower cost."

Some users reported mixed results in practical application. While some found it "incredible for driving out work" that previously required larger models like Claude Opus, others noted that the model can occasionally get stuck in loops or struggle with highly complex, non-obvious vulnerabilities compared to models like GLM 5.3. Specifically, one user noted that while DeepSeek v4.1 Flash performed well on targets with "low-hanging fruits," it struggled more with a fully-annotated decompilation of a 3DS kernel compared to GLM 5.3.