Lecture, Conference Presentation, Fireside Chat
MIT AGI: Building machines that see, learn, and think like people (Josh Tenenbaum)
- Core Thesis: Current AI systems are "AI technologies" (specialized pattern recognition tools) rather than true "Artificial General Intelligence" (AGI) because they lack common sense, flexible general-purpose intelligence, and the ability to model the world beyond data correlation.
- Limitation of Deep Learning: While deep learning has matured pattern recognition (e.g., playing Go), it cannot replicate human abilities like driving, learning chess from a handful of games, or understanding the concept of a "game" without massive, task-specific engineering.
- CBMM Vision: The Center for Brains, Minds, and Machines (CBMM) advocates for "reverse engineering" human intelligence by jointly pursuing basic cognitive science and engineering to build machines that model the world as humans do.
- Scientific Principle: Cognitive science and neuroscience should be approached as engineering endeavors, where the output is not just a descriptive text but a computational model translatable into engineering applications.
- Historical Context: Foundational algorithms for deep learning (backpropagation, reinforcement learning, LSTMs) were originally developed by cognitive scientists and psychologists in the 1950s–1980s, often published in psychology journals, formalizing insights about the mind in mathematical terms.
- 50-Year Roadmap: Long-term goals include solving fundamental questions regarding consciousness, meaning, language, cultural transmission, creativity, and the origins of real learning.
- Visual Intelligence Challenge: Humans construct a rich, high-resolution model of the world from low-resolution foveal data and saccadic eye movements; current systems fail to track objects outside their immediate field of view (e.g., knowing what is behind them without turning).
- Critique of Image Captioning: Industry captioning systems (e.g., Microsoft, Google) exhibit "dataset overfitting," where they reproduce statistical patterns from training data rather than understanding the semantic content or physical reality of an image.
- Evidence: Twitter bots using state-of-the-art captioning systems fail on out-of-distribution data, hallucinating people in empty scenes or misidentifying objects based on biased training associations.
- Human vs. Machine Understanding: A famous photo of Barack Obama pressing a scale to prank someone reveals that human understanding requires modeling physical forces, mental states (intent to deceive), and social context—capabilities current AI lacks.
- Robotic Learning Gap: Industry robots (e.g., Boston Dynamics, Google's arm farm) rely on massive datasets and millions of examples to learn simple physical tasks, whereas human infants learn to manipulate objects with a fraction of data by building internal models of physics and goals.
- Infant Cognitive Evidence:
- Stacking Cups: Toddlers (18 months) demonstrate hierarchical symbolic planning (sub-goals, debugging) and understanding of physical constraints (e.g., moving an obstruction to access a goal).
- Object Permanence: Babies successfully plan actions to retrieve objects they have not seen for extended periods, indicating robust internal representation of the world.
- Intuitive Psychology: Infants (10 months) understand agent goals based on "naive utility calculus" (cost of action vs. reward) and can distinguish between helping and hindering behaviors.
- Technical Approach 1: Probabilistic Programs: The proposed architecture combines symbolic representation, probabilistic inference (to handle uncertainty and sparse data), and neural networks for pattern recognition.
- Implementation: Uses probabilistic programming languages (e.g., Church, Pyro, PyTorch Probabilistic) to encode common sense knowledge as executable programs.
- Technical Approach 2: "Game Engine in the Head": Human cognition may utilize internal simulations similar to video game engines, featuring fast, approximate physics engines and simple AI planning models.
- Intuitive Physics Engine: Models successfully predict human judgments in block-stacking experiments by running low-precision simulations of physical outcomes.
- Intuitive Psychology Engine: Models infer agent goals by running inverse planning (working backward from observed actions) over a physics-based planner (e.g., MuJoCo).
- Learning as Programming: Unlike end-to-end optimization (gradient descent), learning in humans is framed as "programming" or "hacking" internal models to improve accuracy, efficiency, and portability.
- Evidence: The "one-shot learning" of handwritten characters (Lake et al., 2015) demonstrated that machines can learn generative models of characters by inverting probabilistic programs rather than training large datasets.
- Future Frontier: The ultimate goal is to build systems that "grow into intelligence" like humans, starting with a "baby" mind and learning like a child, requiring a convergence of cognitive science, programming languages, and probabilistic inference.
- Industry vs. Academia: Industry currently focuses on 2-year ROI and scalable pattern recognition, whereas academia explores the foundational science of cognition; Josh Tenenbaum argues that bridging this gap is essential for achieving true AGI.
- Hardware Implications: Current hardware is insufficient for AGI; future success requires low-power, high-parallelism computing inspired by the brain (e.g., ~10 billion cores, low energy consumption), potentially driven by advances in the video game industry.
- Neuroscience Integration: Cognitive-level abstractions (software) provide necessary targets for neural circuit research; studying neurons without understanding the computational tasks they perform makes it difficult to identify viable engineering solutions.