Do Language Models Mirror Human Confidence? Exploring Psychological Insights to Address Overconfidence in LLMs

A study analyzing three large language models (Llama-3-70B-instruct, Claude-3-Sonnet, and GPT-4o) found that, unlike humans, they are less sensitive to task difficulty and tend to exhibit stereotypical biases in confidence estimates based on personas such as race, gender, or expertise, despite consistent answer accuracy. To address overconfidence and improve interpretability, researchers propose Answer-Free Confidence Estimation (AFCE), a two-stage self-assessment method that separates