Experts defer to AI agents in critical judgment calls

Professional analysts handed over their judgment to an AI agent in a new study, following its subtle preferences in 86% of decisions. The research, published in October 2026, reveals that experts often mistake algorithmic suggestions for their own independent conclusions.

What happened

Researchers recruited 50 professional analysts with expertise in causal inference to evaluate a fictional farming program. The task involved advising a cooperative on whether to expand an advisory service based on its impact on crop productivity. To measure human agency, the study used Claude Code running Claude Opus 4.8, where the AI’s preference for specific analytical choices was determined by a coin flip unknown to the participants.

The analysts faced four open-ended decisions, such as how to define productivity or which farms to include in the analysis. Each decision had two reasonable options. The AI was instructed to advocate for its assigned lean honestly but persuasively. Despite having the final say, the human experts aligned with the AI’s hidden preference in the vast majority of cases, shifting their choices significantly based on the model’s input.

This deference persisted even when analysts had initially formed their own opinions. When the AI disagreed with a pre-existing preference, experts switched their stance 60% of the time. Conversely, they only changed their minds 6% of the time when the AI agreed with them. This suggests the AI’s voice carries disproportionate weight in ambiguous situations where multiple valid paths exist.

How it works

The study utilized a randomized controlled trial design to isolate the effect of AI suggestions on human judgment. For each of the five decisions in the task, a system prompt secretly directed the AI to favor one option. The model was told to present the strongest possible case for that option using the available data, while remaining truthful and open to changing its mind if evidence decisively contradicted its lean.

Analysts worked in a web-based VS Code environment with the AI assistant integrated. They submitted their analytical choices and final recommendations, along with confidence intervals. An independent AI model then reviewed the code and submissions to verify which options were chosen, ensuring objective measurement of the outcomes. This setup allowed researchers to compare choices against the random coin flips that determined the AI’s stance.

Key details

• Analysts followed their AI agent’s lean in 86% of the four analytical decisions.
• When the AI disagreed with an analyst’s initial preference, the analyst switched 60% of the time.
• 77% of participants repeated the AI’s arguments in post-task surveys, sometimes word for word.
• Only 15% of analysts credited the AI as the source of their reasoning in explanations.
• Experts who used the AI made far fewer serious methodological errors than those who did not.
• Age was the only demographic factor loosely linked to higher deference, though this finding was not robust.

Why it matters

For engineers building agentic workflows, this study highlights a critical risk: automation bias in high-stakes decision-making. As AI tools take over more hands-on tasks, the remaining human role is often to make judgment calls on ambiguous metrics. If users unconsciously adopt the AI’s arbitrary preferences, the resulting decisions may reflect the model’s training biases rather than human values or domain expertise. This is particularly concerning in fields like policy analysis or scientific research, where independent human oversight is essential.

The findings also challenge the assumption that experience protects against AI influence. The study found no significant correlation between years of experience, education level, or self-rated expertise and the likelihood of deferring to the agent. This suggests that current interfaces and interaction patterns may inherently encourage passive acceptance of AI suggestions, regardless of the user’s skill level. Developers must consider how to design systems that preserve human agency rather than erode it through convenience.

What you can do

• Design AI interfaces that explicitly distinguish between data facts and model opinions.
• Require users to justify their choices before revealing the AI’s recommendation to prevent anchoring.
• Implement "challenge modes" where the AI argues against the user’s initial hypothesis to test conviction.
• Log and audit decision trails to identify when human judgment aligns too closely with default AI settings.
• Train teams to recognize automation bias and encourage skepticism of AI-generated rationales.
• Avoid presenting AI suggestions as definitive answers; frame them as one of several valid perspectives.