Interview
DeepMind's Pushmeet Kohli on AI's Scientific Revolution
- DeepMind's AlphaEvolve is an evolutionary AI system that couples large language models (LLMs) with evaluators to discover entirely new algorithms, moving beyond answering questions to actively asking them.
- The system successfully uncovered hidden mathematical truths and generated human-interpretable code that outperforms expert-designed solutions in tasks previously studied for decades.
- AlphaEvolve evolved from FunSearch, which was limited to finding specific algorithmic components within templates provided by researchers; AlphaEvolve removes this constraint to optimize whole algorithms and search across vast code spaces.
- Unlike its predecessor, AlphaEvolve operates with fewer function calls, allowing for faster discovery of new algorithms by evaluating fewer proposals more efficiently.
- The system leverages Gemini Flash for high-speed proposal generation and Gemini Pro for high-quality semantic evaluation, creating a balance between search speed and solution quality.
- AlphaEvolve is language-agnostic, capable of searching for algorithms in C++, Python, and Verilog (for chip design), provided a reliable function evaluator exists for the task.
- The cognitive architecture mirrors the scientific method through a multi-agent setup where one agent acts as a hypothesis generator, another as a critic/reviewer, and a third as an editor/ranker.
- The multi-agent approach effectively extracts insights from the "tail of the distribution," enabling the system to identify non-obvious solutions that a single model might miss.
- In the domain of mathematics, the system improved the known result for 4x4 matrix multiplication from 49 multiplications (Strassen) to 48, a precise result requiring no human feedback (RLHF) for validation.
- For the Cap Set problem, the AI discovered previously unrecognized symmetries in the mathematical structure, a breakthrough confirmed by collaboration with mathematician Terence Tao.
- The technology prioritizes interpretability over black-box neural network policies, generating code that human engineers can debug, verify, and understand, which is critical for data center job scheduling.
- Unlike reinforcement learning agents that optimize for metrics without explainability, AlphaEvolve produces executable programs that allow human experts to recover insights from the AI's logic.
- The system's applicability extends beyond pure math to real-world engineering problems in chip design (Verilog) and data center scheduling, provided a programmatic evaluator can quantify solution quality.
- While human evaluators can be used for subjective criteria like elegance or simplicity, current implementations rely primarily on programmatic evaluators to handle the scale of the search.
- The impact of AlphaEvolve is modeled after AlphaFold 2, which transformed structural biology by democratizing access to protein structures for researchers in regions without access to expensive synchrotrons.
- AlphaFold 2 reduced protein structure prediction time from years to seconds and cost from $1 million to negligible amounts, accelerating drug discovery by removing a primary bottleneck.
- Despite the acceleration in target identification, drug discovery remains limited by clinical trial phases (I, II, III) and toxicity testing, which require further AI advancements to speed up.
- Future societal impacts are predicted to be most profound in energy (e.g., room-temperature superconductors, fusion) and coding (ubiquitous intelligence), potentially reshaping geopolitics and economics.
- Major bottlenecks for the future of AI-driven discovery include validating digital results in the physical world and ensuring the technology is accessible to the broader scientific community.
- A critical gap in current LLMs is the lack of calibrated uncertainty; unlike AlphaFold, which indicates confidence levels, current models do not reliably signal when they are likely to be wrong.
- Pushmeet Kohli expresses a bullish outlook on robotics, specifically humanoid robots, due to the existing infrastructure and architecture of the non-natural world being designed for human form.
- Pushmeet predicts a future where Nobel Prizes are won by human-AI collaborative teams, with AI becoming an indispensable tool rather than a standalone winner.
- The discussion identifies inference-time compute and test-time training (distillation) as the next major frontiers for scaling AI intelligence and efficiency.
- A "must-read" paper highlighted by Kohli is the Wake-Sleep algorithm from MIT (Kevin Ellis, Josh Tenenbaum), which proposes building "libraries" of common modules to facilitate future program generation.