Disinformation & synthetic media on the rise | Nina Schick (2021)

Core Thesis & Context

• The rise of synthetic media (deepfakes) represents an AI-led paradigm change in human creation, content production, and perception, comparable in magnitude to the invention of the printing press.
• Nina Sheik argues this issue is as critical as climate change because the "information ecosystem" encompasses all aspects of human existence and societal function.
• The current information ecosystem is "completely unprecedented in history," combining the internet, smartphones, social media, and synthetic media, none of which were built with safety mechanisms to handle human malice.
• The shift from "age of information" to "age of synthetic media" challenges the assumption that digital proof (video/audio) guarantees truth, potentially returning society to a pre-1900s state of unverifiable claims.

Technological Capabilities & Trends

• Accessibility: The barrier to creating deepfakes has plummeted; as of 2021, voice synthesis can be achieved with just five seconds of audio, and image-based deepfakes are accessible via consumer apps without specialized hardware.
• Evolution: The technology moved from requiring hours of training data (2018) to near-instant generation (2021) following the 2014 breakthrough in Generative Adversarial Networks (GANs) by Ian Goodfellow.
• Ubiquity Forecast: Experts project that up to 90% of audiovisual content online could be synthetically generated by the end of the decade.
• Detection Gap: Current detection models are limited to specific training data and cannot universally identify deepfakes in the wild; it is increasingly viewed as an "adversarial game" where detectors will likely never fully outpace generators.
• Biometric Hijacking: AI can now clone voices and faces of real individuals without their consent, creating risks where digital identities are compromised by anonymous actors.

Primary Harms & "The Liar's Dividend"

• The Liar's Dividend: The mere existence of deepfake technology allows bad actors to deny authentic evidence of wrongdoing (e.g., politicians claiming incriminating videos are fakes), eroding trust in all media, including authentic content.
• Case Study (George Floyd): A PhD candidate and Republican candidate published a 23-page report claiming the George Floyd video was a deepfake; while the claim failed then, Sheik predicts it will gain traction in more polarized future environments.
• Case Study (Gabon): The President of Gabon used a deepfake-style video (due to stroke effects) to prove he was alive after a stroke, inadvertently fueling rumors that he was dead and triggering a coup attempt.
• Case Study (Malaysia): A minister involved in a leaked homosexual affair denied the video was real (claiming it was a deepfake) and avoided punishment, while his partner who admitted the truth was imprisoned.
• Non-Consensual Pornography: This was the first major application of deepfakes (2017); it has evolved from targeting celebrities to targeting "normal" women and minors, often without any legal recourse in any country globally.
• Political Genocide: In Myanmar, "cheap fakes" (manipulated, non-AI imagery) spread on Facebook helped incite genocide against the Rohingya minority after internet access was granted to the junta.
• State Actors: Russia remains responsible for over 70% of state-led disinformation campaigns (2019 data), but China is rapidly increasing its influence operations, including using "cheap fakes" to damage allies' reputations (e.g., the Australian soldier incident).

Societal & Political Implications

• Existential Risk to Democracy: The primary threat is not mass gullibility, but "cynicism" where citizens stop believing any shared objective reality, making democratic consensus impossible.
• Privacy Paradox: While deepfakes allow individuals to deny past actions (potentially protecting privacy or allowing escape from unjust persecution), this primarily benefits the powerful who can deny accountability for crimes.
• Identity Politics: Disinformation amplifies existing societal fractures; as seen in Myanmar, the US (Pizzagate/QAnon), and India, synthetic media can weaponize identity politics to destabilize pluralistic societies.
• Authoritarian Advantage: Authoritarian regimes (e.g., China) may use state-controlled information ecosystems to exert monolithic control over narratives, potentially using deepfake technology to label genuine human rights abuses as "fakes."
• Global Disparity: The next decade will see billions in Africa and India join the information ecosystem without the "digital literacy inoculation" that Western users have developed, making them highly vulnerable.

Solutions & Future Pathways

• Media Provenance: The preferred solution is building a trusted ecosystem via "Content Authenticity Initiatives" (led by Adobe/C2PA) that embed metadata at the point of capture to verify the source and history of media.
• Legislative Challenges: Enacting blanket bans on deepfakes is ineffective due to gray areas like political satire; legislation should focus on "consent" for biometric use and mandatory labeling of synthetic content rather than prohibition.
• Inoculation: Public education and "inoculation" against synthetic media (e.g., interactive quizzes) are necessary but carry the risk of increasing general cynicism.
• Positive Applications: The technology is being used for social good, including restoring voices to those who lost them due to disease (Vocal ID), democratizing content creation for creators, and enabling corporate training in multiple languages (Synthesia).
• Tech Industry Responsibility: Sheik criticizes major platforms for "paying lip service" to the problem; they possess the resources to lead technical solutions (detection and authentication) but have not done so aggressively.
• Career Opportunities: Sheik encourages researchers, engineers, and policymakers to focus on media authentication, detection algorithms, and ethical frameworks, noting that the pace of change requires networked, interdisciplinary solutions similar to climate change efforts.

Specific Recommendations & Resources

• Organizations to Follow:
  • Witness: Focuses on using verified media for human rights documentation.
  • Sensity: A startup specializing in deepfake detection technology.
  • Content Authenticity Initiative (C2PA): The leading coalition for setting global standards for media provenance.
  • Synthesia & Descript: Examples of commercial applications for synthetic media (education, corporate, podcasting).
• Legislative Standard: Laws should mandate that synthetic media be clearly labeled as such, and if used for political satire, it must be distinguishable from reality to prevent the "Liar's Dividend."