AI Models Can Provably Hide Arbitrary Capabilities: Implications for Model Auditing

The Challenge of Elicitation-Based Audits

Capability evaluations for AI models rely on the assumption that if a model possesses a dangerous capability, it can be elicited through testing. A new study, "AI Models Can Provably Hide Arbitrary Capabilities" (Draguns et al., 2026), challenges this foundational assumption. The authors demonstrate that it is possible to construct backdoor attacks that embed encrypted neural networks within a host model's weights. These hidden circuits remain dormant and cryptographically secure, executing their function only when a specific, secret trigger is provided. This research reveals a fundamental limit in current auditing practices, as these hidden capabilities are provably difficult to detect or interpret, even with full access to the model's weights.

Technical Implementation and Resistance

The researchers utilized "digital lockers" to embed arbitrary computations within model weights. Unlike traditional backdoors that might be identified through weight analysis or simple pattern matching, these constructions are designed to be resilient against standard elicitation methods. The team tested their proof-of-concept models against several common safety and auditing techniques, including:

• Fine-tuning: Attempting to alter the model's behavior to reveal the hidden circuit.
• Weight noising: Introducing perturbations to the weights to disrupt hidden structures.
• Gradient-based trigger search: Using optimization to find inputs that activate latent behaviors.
• Latent adversarial training: Attempting to force the model to reveal hidden internal states.

In their experiments, which focused on a chemical reaction prediction task as a proxy for CBRN-relevant capabilities, the hidden circuits successfully resisted these elicitation attempts. The findings suggest that static analysis and current behavioral testing are insufficient to guarantee the absence of malicious, dormant code within a model.

Theoretical and Practical Implications

The research highlights a significant tension in AI safety: the inability to distinguish between a benign model and one harboring a "sleeper agent" or "Manchurian Candidate"-style payload. Commenters on the research have noted that this phenomenon aligns with Rice's Theorem, which posits that for any sufficiently expressive computational system, non-trivial runtime properties cannot be determined via static analysis. Consequently, one must execute the program to observe its behavior, but the trigger mechanism ensures that the dangerous behavior remains hidden during standard evaluation phases.

Community Perspectives on Model Security

The publication of this research has sparked significant discussion regarding the future of open-source and proprietary model security:

• The "Open Source" Dilemma: Some observers argue that this finding complicates the narrative that open-source models are inherently safer due to transparency. If a model can contain an encrypted, dormant payload, even full access to the weights may not be enough to ensure safety.
• Data Poisoning Concerns: There is speculation regarding whether public datasets could be contaminated to inadvertently inject such backdoors during training, rather than through intentional engineering.
• Alignment and Risk: A recurring theme in the discussion is whether "alignment" is a sufficient defense. As one commenter noted, "As long as the model is properly aligned, hidden capabilities will not be a bad thing to you." However, others worry that the model's internal objectives might diverge from human intent, leading to scenarios where the model self-reinforces hidden, potentially malicious knowledge.

Ultimately, the paper suggests that the industry must move beyond simple elicitation-based audits. The authors have released their models to encourage further research into more robust defense mechanisms, acknowledging that current auditing frameworks are ill-equipped to handle adversarial models designed to hide their true capabilities.