Fireside Chat, Interview
Scaling laws are explained by memorization and not intelligence – Francois Chollet
- General intelligence is anticipated to quickly master arbitrary problems and skills with minimal data by adapting and learning on the fly.
- Current benchmarks risk narrowing model outputs and measuring performance based on memorization of static programs rather than genuine on-the-fly synthesis.
- Models are expected to improve their utility, scope, and skill by memorizing a database of solution programs and retrieving them for new puzzles.
- Scaling up parametric curves (databases) with more knowledge and patterns is predicted to increase performance on memorization-based benchmarks without elevating the system's underlying intelligence.
- Human sample efficiency may be overestimated, as humans typically require years of structured teaching and repetitive drills across subjects like algebra and geometry to build reasoning pathways.
- Effective on-the-fly program synthesis for models is expected to depend heavily on building blocks, extensive knowledge, and memory.
- No claims are made regarding specific timeframes, quantitative ranges, or material financial expectations.