Advice presented as coming from artificial intelligence was linked to lower confidence in a 440-person experiment, with a larger decline among participants who rejected it.
The research, published in Decision Support Systems on 22 August, examined how people assessed their own judgements when a computer recommendation disagreed with them. It suggests that accepting an answer and feeling influenced by it are separate responses.
There is a significant limit to the finding. The advice was scripted for a money-sharing experiment and labelled as AI. The study did not test a working chatbot or follow managers making decisions in their jobs.
A decision could stay the same while confidence fell
Researchers Leily Soleimanof and Derrick Neufeld, of Ivey Business School, examined participants’ predictions about another person’s behaviour.
After a short conversation, each participant decided how much money to give the other player. They then predicted how much the other person had given them, rated their confidence and saw a purported AI estimate before making a final prediction.
The researchers’ published study materials explain that the estimate was deliberately set above or below the participant’s initial prediction. It was not produced by an algorithm analysing their conversation.
Confidence was measured before and after that advice. The paper reports lower confidence afterwards, with a more pronounced decline among people who rejected the recommendation.
This means a person could retain their original answer while becoming less certain about it. The result does not, by itself, show that the answer became worse.
Knowing when to trust your own judgement
The authors also examined metacognitive sensitivity: how well a person’s confidence matched the accuracy of their judgement.
Among participants whose confidence and accuracy were aligned, those with low confidence were more likely to move towards the advice. Those with high confidence were more likely to hold their ground.
Supplementary results show that 48% of the first group adjusted their predictions towards the recommendation, compared with 14% of the second. Where confidence and accuracy were poorly aligned, advice-taking differed less between high- and low-confidence participants.
These are comparisons within the experiment. The published protocol included no group receiving no advice, which limits what the before-and-after confidence comparison can establish. The findings do not show that training employees to feel more confident would improve their decisions.
What it leaves open for employers
The findings add a separate perspective to the Bath-led research on AI shortcuts and managerial judgement that MBN recently covered.
That theoretical paper proposes that time pressure can encourage managers to hand over too much thinking to generative AI, while accountability can encourage them to question its answers. It did not measure a decline in managers’ abilities.
Bath professor Dirk Lindebaum said managers could use AI to challenge assumptions and test their reasoning, provided they put effort into examining gaps in its explanations.
For employers, the confidence experiment raises a further question: whether staff can judge when their own answer deserves trust. It does not establish how long the reported change in confidence lasts, or whether the same pattern appears in everyday managerial work.
