Access to AI advice suppresses critical thinking and makes people more confident when repeating incorrect information — even when the AI is known to be unreliable, according to a new study published by researchers at the University of Milano-Bicocca, Ecole Normale Superieure, and Sapienza University of Rome.
The study, titled “AI advice suppresses people’s willingness to say ‘I don’t know,’ even when the advice is wrong and accuracy is incentivized,” tested participants on visual detail questions about films. One group answered without AI assistance; another could consult a language model chosen deliberately because it was usually wrong on those specific questions.
The results were stark. In the control group, 44 percent of answers were “I don’t know” — a measure of intellectual humility. Among participants with access to AI, only 3 percent said they didn’t know, even though their accuracy dropped from 27 percent to 9 percent. Confidence in being correct more than doubled, from 30 to 76 percent.
“For humans, the capacity to say, ‘I don’t know,’ is very important because it represents the recognition of the limits of our own knowledge,” said Valerio Capraro, the study’s lead author. “But now with AI, we can get an easy answer to virtually every question, so we wondered whether this would interfere with human capacity to say, ‘I don’t know.'”
Monetary incentives for correct answers improved the outcomes only marginally — the “I don’t know” rate rose to 8 percent and accuracy to 16 percent, still far below the unaided baseline.
Capraro said the findings apply beyond trivia questions. “I’m very much concerned for children,” he said. “Adults have learned critical thinking. But for children who basically are born with these systems, the risk is that they don’t even learn the basic critical skills.”
The researchers recommend AI literacy education as the primary intervention, noting that model providers have limited incentive to solve the problem themselves.
Sources: Using AI makes people less likely to admit they don’t know something (The Register, Jul 19)