Interview
Yann LeCun: Deep Learning, ConvNets, and Self-Supervised Learning | Lex Fridman Podcast #36
- The convergence of lawmaking and computer science is anticipated to produce objective functions aligned with the common good, alongside hardwired constraints akin to a Hippocratic Oath to prevent specific actions in future autonomous systems.
- Architectural evolution for reasoning systems is expected to require a "working memory" for episodic facts, iterative processing for state updates, and new memory designs capable of scaling to Wikipedia-sized associative datasets.
- Training methodologies will increasingly shift toward interactive environments like robotics simulations or games, while self-supervised learning is identified as a necessary prerequisite for all other learning forms, including reinforcement and supervised learning.
- Progress in natural language processing via self-supervised learning is projected to continue, though significant challenges remain in image recognition and video prediction due to representing uncertainty in continuous visual spaces.
- Active learning is predicted to improve efficiency in tasks like autonomous driving without driving a "quantum leap" in machine intelligence, with long-term solutions relying on a combination of self-supervised and model-based reinforcement learning rather than engineering-heavy sensor fusion.
- The term "Artificial General Intelligence" is expected to be abandoned in favor of recognizing human intelligence as highly specialized, while human-level systems are predicted to initially lack high-level intelligence, potentially resembling a four-year-old child.
- Full-fledged autonomous systems are planned to consist of four components: a hardwired contentment objective calculator, a contentment predictor, a world model, and a policy network, with the necessity of "emotion" defined as anticipating bad outcomes.
- Common sense reasoning is expected to emerge from grounding in physical reality through language interaction, visual input, and virtual or real environment interactions, with the subjectivity of objectives for such systems becoming relevant only once the technology exists.
- Achieving human-level intelligence faces a primary obstacle in the form of the "mountain" of self-supervised learning, which must be overcome before higher cognitive capabilities can be developed.