Overconfidence and its Societal Impact

Key Insight

Overconfidence is a pervasive cognitive bias and a direct consequence of the WYSIATI (What You See Is All There Is) rule, where subjective confidence is driven by the coherence of the story constructed from available information, making it impossible to account for what is unknown or absent. When individuals estimate quantities or make forecasts, they rely on information that comes easily to mind, leading to exaggerated beliefs in their own abilities.

This overconfidence is starkly illustrated by a study of chief financial officers (CFOs) of large corporations, who made 11600 forecasts of the S&P index. The correlation between their estimates and actual returns was slightly less than zero, yet they were grossly overconfident, with 67% of outcomes falling outside their 80% confidence intervals—more than three times higher than expected. To be realistic, their confidence intervals would need to be four times wider, a 'confession of ignorance' that is socially unacceptable for experts paid to be knowledgeable.

Overconfidence has costly societal consequences, as organizations taking advice from overly confident experts are prone to excessive risk-taking, contributing to issues like the financial crisis where collective blindness to risk prevailed. Professionals, including pundits and physicians, are often encouraged by clients and society to display high confidence, even when wrong (e.g., clinicians 'completely certain' of a diagnosis were wrong 40% of the time). This social pressure means experts who acknowledge their ignorance may be replaced by more confident competitors, perpetuating a culture where acting on pretended knowledge is often preferred over an unbiased appreciation of uncertainty. However, methods like the 'premortem,' where a team imagines a future disaster to identify potential threats, offer a partial remedy by encouraging doubt and unleashing imagination.