How to spot an AI Deepfake

Current Attack Landscape

• Approximately 90% of security attacks still target people rather than systems, contradicting the ego-driven belief among technology leaders that systems alone can be secured.
• Social engineering attacks have increased over 4x since the release of ChatGPT two years ago.
• The United States experienced over 100,000 deepfake attacks in 2024, with exposure rates rising from 5-10% of the population a year ago to an estimated 30-40%.
• Open-source models like DeepSeek allow adversaries to run sophisticated attacks on consumer devices globally, removing previous cost prohibitions for brute-force campaigns across voice, SMS, video, and chat channels.
• Adversaries are increasingly using AI agents to automate and scale social engineering operations, enabling single attackers to act as infinite autonomous teams.

Specific Threat Vectors and Tactics

• Virtual Kidnapping: AI-generated voice replication has enabled criminals to mimic family members (e.g., a father's voice) in distress to demand immediate wire transfers.
• Supply Chain Infiltration: State-sponsored actors (e.g., North Korean operatives) are successfully entering Silicon Valley tech companies via remote job applications, bypassing background checks to access and exfiltrate code and secrets before disappearing.
• Geopolitical Espionage: Deepfakes are being used to impersonate government officials (e.g., Russian delegates) to extract sensitive information from U.S. political figures regarding international conflicts.
• Impersonation Tactics: Scammers intentionally introduce typos and grammatical errors in communications to appear more authentic and drive engagement with victims.
• Critical Infrastructure Risk: While current losses are primarily financial, there is a high likelihood of future large-scale attacks targeting hospitals and energy systems, posing physical threats to human life.

Corporate Vulnerabilities and Training Gaps

• Authority Bias: Employees are uniquely vulnerable in enterprise settings due to the overwhelming psychological power of authority figures (e.g., CEO impersonations) to bypass critical thinking.
• Legacy Training Inefficacy: Standard compliance training is often viewed as a "check-the-box" exercise, is updated infrequently, and lacks relevance, resulting in poor ROI and low employee retention of security concepts.
• Human System Lag: Technical email security has advanced, but human defenses remain the weakest link, with significant exposure in non-email channels that lack automated guardrails.
• Security Posture Signal: Boring or obsolete security training signals to employees that the organization does not value security, reducing vigilance and encouraging risky behavior.

Recommended Defensive Strategies

• Immediate Consumer Actions: Individuals should delete personal voicemail greetings containing their voice, as these samples are sufficient for AI voice replication.
• Behavioral Adjustments: Users should avoid providing audio samples to unknown callers; responding with only "hello" without confirming identity limits the data available for deepfakes.
• Enterprise Education Shift: Training must move from annual compliance to continuous, personalized, and "jarring" educational experiences that simulate real-world AI attacks.
• Adaptive Simulation: Organizations should implement ongoing, AI-powered attack simulations (e.g., deepfake calls, personalized phishing) to harden employee resilience and identify specific organizational vulnerabilities.
• Open Source Intelligence (OSINT) Awareness: Companies must educate employees on the extent of publicly available data regarding themselves and their peers, which attackers leverage to craft credible pretexts.
• Technology Development: Investment is needed in AI agents designed for defense that can operate autonomously to counter attacking AI agents, creating a defensive "Co-pilot" for security.

Forward-Looking Outlook

• AI-powered social engineering is predicted to become a normalized, daily consideration for the general public within five years.
• Deepfake and AI-enabled attacks are expected to transition from financial fraud to catastrophic impacts on critical infrastructure and human safety within the next two years.
• The security industry anticipates an arms race lasting over a decade, with defenders and adversaries constantly evolving AI tactics and counter-tactics.
• Resources for ongoing education include the a16z blog on "16 things to protect yourself" and the Adaptive Security corporate blog, which tracks real-time attacks and trends.