Interview, Podcast
Why human gullibility and misinformation are overrated | Hugo Mercier
- Core Thesis: Human beings possess evolved mechanisms of "open vigilance" that generally prevent widespread gullibility; people are skeptical by default and only accept new information when it passes plausibility checks or comes from trusted sources.
- Evolutionary Logic: Communication requires mutual benefit; receivers evolve to be discerning because ignoring false signals is safer than acting on them. In contrast, species with adversarial interests (e.g., mother vs. fetus) lack honest signaling mechanisms.
- Intuitive vs. Reflective Beliefs: People distinguish between "intuitive beliefs" (which drive behavior and are heavily scrutinized) and "reflective beliefs" (often held but not acted upon).
- Conspiracy Theories: Vocal conspiracy theorists (e.g., claiming CIA orchestrated 9/11) are less likely to be correct because they face no fear of retribution; actual insiders with "intuitive" knowledge of conspiracies (e.g., government shredding documents) tend to remain silent to avoid job loss or death.
- Social Utility: Reflective beliefs often persist because they confer social status, group belonging, or signal allegiance (e.g., "burning bridges" to prove loyalty to a radical group).
- Trust Mechanisms: Trust is evaluated along two dimensions:
- Competence: Does the source know the truth? (Determined by expertise, informational access, and track record).
- Honesty: Will the source tell the truth? (Determined by short-term incentives and long-term alignment of interests).
- Systematic Errors:
- Plausibility Checking: People reject information that clashes with prior beliefs unless it comes from a trusted authority or a strong argument.
- Information Cascades: People often misinterpret the independence of others' beliefs; if a few people adopt a view, others may follow without realizing the source is the same.
- Regression to Mean: Health treatments (e.g., homeopathy, bloodletting) are often adopted because ailments naturally improve over time, creating an illusion of efficacy.
- Teleological Intuition: Beliefs like creationism or astrology persist because humans naturally infer purpose in nature, making random chance counter-intuitive.
- Vaccine Hesitancy: A universal, cross-cultural phenomenon driven by the counter-intuitive nature of injecting a transformed disease agent into a healthy body, not merely by disinformation.
- AI and Misinformation Risks: Hugo Mercier argues that fears regarding AI-generated misinformation are largely exaggerated due to specific bottlenecks:
- Inundation (Quantity): The bottleneck is attention, not content production; flooding the zone with articles does not increase persuasion if no one reads them.
- Silver-Tongued AI (Persuasion): Increased writing quality does not override the human tendency to shut down on complex arguments that contradict strong priors.
- Scalpel AI (Personalization): Hyper-personalized arguments fail to scale because persuasion relies on social transmission, which requires content to be shareable among groups, not just tailored to the individual.
- Spam (Noise): The only plausible risk is "mass indifference," where an influx of fake content causes people to give up on forming any strong views rather than changing specific ones.
- Mitigation Strategies:
- Reputation Systems: Truth is maintained by institutions (journalists, academics) that risk reputation for falsehoods; AI cannot easily replace the "skin in the game" required for reliable curation.
- Official Misinformation: Lies endorsed by government officials or prestigious academics pose a greater risk than social media bots and must be aggressively debunked.
- AI as Defense: AI can be used to automate fact-checking, generate counter-narratives ("information bots"), and flag misleading accounts.
- Historical Data: The World Economic Forum ranked misinformation as the #1 global risk over the next two years (ahead of war and AI), a ranking Mercier views as exaggerated.
- Case Studies on Gullibility:
- Milgram Experiment: High compliance was driven by specific cues (Yale prestige, scientific justification) rather than blind obedience; many participants suspected the setup.
- Zimbardo Prison Experiment: Described by Mercier as a "scam" with methodological flaws that should be removed from psychology textbooks.
- Financial Decisions (Day Trading/MLMs): Often driven by gambling instincts and the "reporting bias" where successes are publicized and failures are hidden, rather than external persuasion.
- Political Beliefs: Voters often hold false beliefs about complex policy issues because the personal cost of being wrong is negligible (the "rational ignorance" effect), but this is less damaging to democracy than a dictator holding the same beliefs.
- Anecdote: Mercier was briefly persuaded by a false claim that soccer goalposts were being fitted with elastic bands to increase goal-scoring, noting that his judgment was temporarily impaired by being intoxicated.