Conference Presentation, Panel, Fireside Chat, Interview
Google's Jeff Dean on the Coming Transformations in AI
- Large models are expected to solve an expanding range of problems annually, with a future market consolidated into a handful of general-purpose foundational models due to high infrastructure costs.
- Progress will rely on both algorithmic improvements and hardware scaling, where techniques like distillation create lighter models and new approaches train larger systems at unchanged compute costs.
- Multi-modality will enable systems to process and output diverse data types, including text, code, audio, and video, while specialized hardware accelerators for reduced precision linear algebra will require iterative generation improvements.
- Inference hardware efficiency is projected to increase by factors of 10, 20, 50, or 1,000 compared to current standards, with the "Ironwood" TPU generation arriving imminently and requiring high-speed networking for large-scale computation.
- Robotics and agents will rapidly advance, with physical robots performing 20 useful tasks in unstructured environments within the next year or two, eventually cost-engineered to be 10 times cheaper and capable of 1,000 tasks.
- Virtual agents are forecast to reach junior engineer proficiency within approximately one year, managing tasks in virtual environments, running tests, and debugging while occasionally consulting humans.
- Scientific discovery will accelerate as AI-driven approximations of computational simulators become 300,000 times faster, fundamentally altering research methodologies and enabling new connections between data points.
- Software delivery will evolve through the Pathways system, allowing single Python processes to run across thousands of devices, while edge devices will increasingly run highly capable models in low-power environments.
- Future development will shift toward organic, continuous learning systems and rethought traditional algorithms that account for network and memory bandwidth realities, potentially resulting in 10,000-fold compute differences between problem types.
- Educational and information retrieval tools are expected to reach billions of users, with browser-based and desktop agents observing user activity to assist in daily tasks, while legacy cloud stack integration efforts aim to eliminate existing friction.