Machine Unlearning: Making AI Forget | RAISE Summit 2024 | Paris

Market Problem & Regulatory Context

• Training AI models is currently a "one-way track" where data acquisition is possible, but data deletion is inherently impossible in standard architectures.
• Stanford research indicates that 83% of foundational models are non-compliant with the new EU AI Act.
• Risk Factors identified include:
  • Presence of personal data (emails, IDs, geolocation) leading to ethical violations and high fines.
  • Copyright infringement disputes, exemplified by the New York Times lawsuit and artist claims.
  • Internal data leakage where models trained on data from one department expose confidential information from another.
  • "Poison data" inclusion, such as exposed SSH keys, passwords, or instructions on creating harmful items.

Limitations of Current Safeguards

• Guardrails and input/output filters are insufficient as they only act as "Band-Aids" rather than removing underlying data.
• Bypass Evidence:
  • Google researchers extracted raw training data from ChatGPT by prompting it to repeat a word poem 70 times.
  • A Hugging Face chatbot was compelled to expose SSH keys and passwords.
  • LLMs were successfully tricked into providing bomb-building instructions via ASCII art obfuscation.

Hirundo's Core Solution

• Value Proposition: The first machine learning platform dedicated to finding and removing specific data from trained models (LLMs and classical models) without full retraining.
• Efficiency Metrics:
  • Unlearning operations take minutes compared to the days/weeks required for retraining.
  • Cost and computational resource usage are <5% of full retraining or fine-tuning alternatives.
  • Significantly reduces GPU usage and carbon footprint.
• User Benefits:
  • AI Engineers: Faster experimentation cycles and improved model accuracy.
  • Compliance Officers: Ensures adherence to GDPR and EU AI Act.
  • CISOs: Eliminates backdoors and sensitive data exposure in LLMs.

Technical Methodologies

• Neuro-Surgical Approach (Weight-Based):
  • Locates specific parameters and weights within billions of model parameters where target data is stored.
  • Adjusts these weights to remove the data's imprint while maintaining the model's functional generalization.
• Data-Influence Approach (Data-Based):
  • Degrades the model's perception of the target data.
  • Amplifies/reinforces the perception of remaining data based on influence analysis to preserve overall performance.

Demonstrated Efficacy

• Case Study: Fine-tuned Gemma 7b model was trained with Ben Luria's email address (ben@irundo.io).
• Result: After the unlearning process, the model immediately "forgot" the email and began hallucinating unrelated career information, proving data removal without guardrail manipulation.

Company & Strategic Outlook

• Funding Status: Pre-seed stage backed by Alpha Intelligence Capital and Supersede (UK).
• Leadership Team:
  • Chief Scientist: Former Dean of Computer Science at the Technion (Israel).
  • CTO: Present for technical inquiries.
• Market Positioning: Addresses the scalability issue of AI regulation where models face millions of user "right to be forgotten" requests daily.
• Forward-Looking Statement: As AI regulation tightens, the inability to selectively remove data makes retraining unrealistic; unlearning is positioned as the necessary operational standard for future compliance.