How AI agents shape experts’ judgment calls

Executive Summary

When experts use AI agents to assist with complex analytical tasks, they often defer to the agent's preferences on subjective judgment calls, even when those preferences are arbitrary. In a controlled study, 50 professional analysts using Claude Code adopted the agent's randomly assigned "lean" 86% of the time, often internalizing the agent's reasoning as their own without recognizing the source of the influence.

Study Design and Methodology

To test the extent of human-AI deferral, researchers recruited 61 analysts with expertise in causal inference. Participants were tasked with advising a fictional farming cooperative on whether to expand a productivity program. The task required four specific analytical decisions—such as choosing a productivity metric or a statistical comparison method—where both options were defensible.

Researchers used a randomized controlled trial design where each participant's AI agent was given a hidden "lean" for each decision, determined by a coin flip. The agent was instructed to make the strongest case for its assigned option while remaining honest and evidence-based.

Key Findings on Deferral

High Rates of Alignment: Across all four analytical decisions, analysts' choices matched their agent's assigned lean 86% of the time.
Internalization of Reasoning: In post-task surveys, 77% of analysts used the agent's specific arguments to justify their choices, yet only 15% credited the agent as the source.
Illusion of Agency: Despite the high rate of alignment with the agent's lean, analysts reported similar levels of confidence in their final recommendations as those who did not follow the agent's lean, and they frequently described the results as their own judgment.
Predictive Power of Agent Leans: The agents' assigned preferences were strong predictors of the final outcome. A statistical model based solely on the four coin-flip-driven decisions correctly predicted 82% of the analysts' final recommendations.

Impact on Expert Autonomy

While AI assistance helped analysts work faster and reduced the frequency of serious methodological errors, the study highlights a significant risk: the erosion of human agency. When experts unknowingly adopt an agent's arbitrary preferences, the resulting work may lack the independent human judgment typically expected in scientific and policy-making contexts.

This phenomenon of "disempowerment" is particularly concerning for AI safety research, where human reviewers are expected to provide an independent check on AI-generated results. If the human reviewer's judgment is influenced by the same AI model they are auditing, the independence of that check is compromised.

Implications for Future AI Integration

Measuring Deferral: Because deferral is difficult to detect, researchers need new methods to estimate how an expert would have performed without AI assistance to identify when they are being swayed by model bias.
Agent Design: Future agent architectures could be designed to present multiple viewpoints or explicitly ask for the user's preference before offering a recommendation, rather than defaulting to a single, potentially biased stance.
Institutional Oversight: Organizations must move away from assuming that an expert's signature implies independent, critical judgment. Institutions should explicitly define which decisions require human-only deliberation and ensure that human skills are maintained even as AI takes on more of the analytical workload.