Sequoia Capital52 min
Cracking the Code on Offensive Security With AI ft XBOW CEO and GitHub Copilot Creator Oege de Moor
Oege de Moor, Konstantine Buhler, Sonya Huang
Interview, Fireside Chat
Market Context & Disruption Thesis
- AI code generation is increasing vulnerability density as models trained on public source code (containing existing flaws) produce more insecure code.
- Attackers are simultaneously leveraging AI to enhance the effectiveness of malicious campaigns.
- This dual threat (more vulnerable code + smarter attackers) necessitates the automation of offensive security, a niche previously limited by high costs and slow turnaround.
Expo Product Performance & Benchmarks
- Initial product testing on industry-standard benchmarks (sourced from PortSwigger and Pentastellan) yielded a 75% success rate.
- Skepticism regarding the 75% score arose due to concerns that benchmarks were present in the training data; consequently, Expo created a proprietary set of original benchmarks not in any training set.
- On these new, proprietary benchmarks, Expo achieved an 85% success rate.
- In a comparative test against five professional penetration testers (including a top-tier expert securing multi-billion dollar hedge funds) on 104 challenges:
- The top human expert scored the same 85% as the AI.
- The human expert required 40 hours to complete the test.
- The Expo AI system completed the same test in 28 minutes.
Real-World Vulnerability Discovery
- During early design partner engagement, the system autonomously discovered a critical vulnerability in a major financial institution (a client with massive security budgets and constant human testing).
- The specific flaw involved an insecure signing algorithm in PHP source code that allowed unauthorized access to a secondary host.
- The AI subsequently identified a vulnerability in JavaScript allowing the download of password files.
- The discovery process demonstrated "human-like" creative exploration, such as iteratively testing parameters and chaining discovered attacks across different endpoints.
Operational Model & "Service as a Software" (SaaS)
- Expo automates the entire simulated attack process, enabling continuous security testing rather than the traditional annual or bi-annual manual penetration test.
- The typical cost of a manual penetration test is approximately $18,000, limiting frequency; Expo aims to replace this with a continuous, automated model.
- Pricing Strategy:
- Base tier: Subscription license based on engineering team size (similar to GitHub Advanced Security).
- Add-on tier: "Attack hours" model allowing customers to pay for increased compute/inference time for more exhaustive testing.
- Guardrails: The system utilizes cloud-only deployment to enforce scope restrictions, requiring customers to prove legitimate ownership of targets to prevent unauthorized attacks.
- Validation: All AI findings are automatically validated for reproducibility and correctness before human review.
Technical Architecture & Data Strategy
- Core Technology: Proprietary security domain knowledge and tooling built on top of foundation models, rather than relying solely on model capabilities.
- Key Components:
- Custom LLM-driven web browser for interacting with targets.
- A curriculum of benchmarks used to train and evaluate the model's progression.
- "Brilliant teenager" guardrails to prevent destructive actions (e.g., dropping database tables).
- Automated attack surface discovery and prioritization tools.
- Data Generation: The team mines open-source repositories (e.g., Docker Hub) to generate new training data by observing successful attack patterns across hundreds of attempts.
- Autonomy: Recent testing showed the AI could identify vulnerabilities (e.g., data leakage) without specific instructions, relying on its contextual understanding of the application's purpose.
Founder Background & Strategic Vision
- Uge Demore (CEO/Founder) is a former Oxford computer science professor and the creator of GitHub Copilot; he previously founded Semmel (acquired by GitHub).
- Timeline: The concept of autonomous agents fixing bugs was theorized in a December 2020 memo during the early Co-pilot development phase.
- Competitive Landscape: Demore notes that while coding assistants are crowded, the opportunity lies in targeting non-professional coders (e.g., Replit model) and high-impact security markets.
- Strategic Goal: The primary driver is to deploy defensive AI faster than adversarial state actors or bad-faith entities can weaponize it.
Future Roadmap & Outlook
- Short-term (Next Few Months): Continued operation with human supervision for early design partners to refine guardrails.
- Medium-term (By Next Summer): Full deployment without supervision, aiming to significantly transform web security standards.
- Platform Expansion: Plans to demonstrate capabilities on platforms like HackerOne and through open-source projects.
- Broader Market Impact: Anticipated disruption in customer support, social media moderation, and long-term expansion into health and biology.
- Long-term Vision: Demore predicts a shift where "English + diagrams" becomes the primary coding language, with AI handling the implementation details, moving up the abstraction ladder from machine language to natural language.
Lightning Round Insights
- Favorite Startups: Suno (AI music generation) and Harmonic (AI mathematical reasoning).
- Advice for Founders: Focus on a single objective and prioritize speed of execution above all else.
- Optimistic Forecast: The next decade will be defined by AI expanding global health outcomes in biology and healthcare.