Jul 3, 2025, 10:00 AM ETConference Presentation, Keynote

François Chollet: How We Get To AGI

Fluid Intelligence vs. Static Skills: Intelligence is defined as the ability to handle novel situations and synthesize new programs on the fly, distinct from static, memorized skills or task-specific automation.
The Failure of Pre-training Scaling: Despite a 50,000x increase in pre-training data and compute, models failed to improve on the Abstraction Reasoning Corpus (ARC-1), scoring only 10% compared to human scores above 95%.
Paradigm Shift (2024): The field has moved from pre-training scaling to Test-Time Adaptation (TTA), where models dynamically modify their internal state during inference to solve unseen problems.
The Role of Benchmarks: Traditional benchmarks (like exams) measure memorized skills and lead to "Goodhart's Law" (optimizing the metric while missing the point); they are inadequate for measuring genuine AGI.
Efficiency as the Metric: Intelligence is defined as the operational efficiency ratio of converting past information (experience) into actions for future, novel situations.

ARC-1 (Released 2019):
- Designed to distinguish fluid intelligence from pattern memorization using 1,000 unique tasks based on core priors (physics, geometry, counting).
- Demonstrated that scaling pre-trained models yields near-zero accuracy on fluid reasoning tasks.
- OpenAI's o3 model (fine-tuned for ARC) achieved human-level performance, validating the TTA paradigm.
ARC-2 (Released March 2025):
- Focuses on compositional generalization and deliberate reasoning rather than instant pattern matching.
- Human Performance: Solved by 100% of a random sample of 400 non-expert humans; requires no specialized training.
- AI Performance: Static models score 0%; single-chain-of-thought models score 1–2%; only TTA systems achieve meaningful scores, yet they remain far below human levels.
- Status: ARC-2 saturates below human-level intelligence, proving current systems lack true general fluid capability.
ARC-3 (Planned for Early 2026):
- Will assess agency, requiring models to explore unknown environments, set goals, and interact autonomously without prior knowledge of controls or objectives.
- Introduces strict action efficiency limits to measure how quickly a model solves tasks compared to human speed.
- Will feature a developer preview in July 2025.

Kaleidoscope Hypothesis: The universe contains limited "atoms of meaning" (isomorphisms) that are recombined to create the appearance of infinite novelty. Intelligence is the ability to identify and reuse these building blocks.
Two Types of Abstraction:
- Type 1 (Value-Centric/Continuous): Based on perception, intuition, and gradient descent; excellent for pattern recognition but fails at discrete logic (e.g., sorting).
- Type 2 (Program-Centric/Discrete): Based on exact structure matching, graph isomorphism, and logical reasoning; essential for human-like problem solving and invention.
The Missing Link: Current deep learning models excel at Type 1 but lack Type 2 capabilities; they cannot perform discrete program search or compositional generalization efficiently.

System 1 + System 2 Integration: The next generation of AI must combine Type 1 (fast, approximate intuition) with Type 2 (rigorous, discrete search).
Deep Learning-Guided Search: Using neural networks to map discrete program spaces into continuous latent spaces to mitigate combinatorial explosion during program synthesis.
Programmer-Like Meta-Learners: Future systems will synthesize custom software (blending DL submodules and algorithmic modules) for each new task rather than relying on fixed parameters.
Dynamic Abstraction Library: Models will maintain a growing, evolving library of reusable code blocks/abstractions, uploading new solutions back to the library for future reuse (analogous to GitHub).
Goal of Endia: To build systems capable of independent scientific invention and discovery, moving beyond automation to accelerate the timeline of science by solving ARC-1 and beyond.