Interview, Podcast
Why human gullibility and misinformation are overrated | Hugo Mercier
- Global risks related to misinformation and disinformation are projected to be ranked as the number one threat over the next two years, a shift in focus from 2023 driven by concerns that generative AI and future super-persuasive AI could fundamentally alter the landscape.
- Search volume for terms related to misinformation and disinformation has increased to five times the level observed in 2017, though some experts argue that fears regarding AI-driven disinformation may be exaggerated.
- The primary bottleneck for AI-generated misinformation is identified as audience attention rather than content volume, with "inundation" strategies deemed unlikely to succeed because people do not easily read new content.
- While "silver-tongued" AI is predicted to be no more effective than humans at changing minds due to existing skepticism, "scalpel" AIs offering personalized arguments may offer slight advantages but face similar scaling challenges.
- There is a risk that "spam" AI content could lead to mass indifference, where individuals abandon the formation of strong views, whereas in low-trust or authoritarian societies, flooding the zone with noise may remain a more effective control strategy.
- Human cognitive evolution is described as making people generally hard to influence, with mass persuasion almost never leading to significant changes in public opinion and the "backfire effect" being rare with only two documented cases.
- People tend to change their minds when presented with good arguments or trusted authority, whereas being distracted or tired results in no change rather than increased susceptibility to persuasion.
- Myths regarding subliminal influence and brainwashing via fatigue or coercion are characterized as having no evidence, and mistakes in trust are argued to be more often the result of not trusting enough rather than credulity.
- The persistence of false beliefs, such as vaccine hesitancy or creationism, is attributed to intuitive heuristics like teleological reasoning and the counterintuitive nature of medical treatments, rather than an inability to distinguish truth from fiction.
- Reflective beliefs, including conspiracy theories, may persist because they provide social benefits like status or in-group bonding, and vocal conspiracies are often less likely to be true because proponents risk safety if the claims were fact.
- Trust in information is asserted to rest on the reputation of institutions and the availability of multiple competing sources, rather than the technical impossibility of faking content.
- A loss of institutional credibility is predicted to result in mass stubbornness and a cessation of changing one's mind, rather than a surge in mass persuasion.
- "Official misinformation" from governments and major institutions is considered to have a significantly larger impact than social media botnets.
- Large-scale AI tools are suggested to potentially improve information quality by acting as automated fact-checkers or identifying misleading accounts, countering the threat of misinformation bots.