Interview
Juergen Schmidhuber: Godel Machines, Meta-Learning, and LSTMs | Lex Fridman Podcast #11
- A theoretically optimal universal problem solver exists in principle (e.g., Gödel machine, Marcus Hutter's "fastest way" algorithm) and can solve all solvable problems, but it remains practically infeasible for current human-scale and small-biosphere tasks due to additive constant overheads; a future practical theory is anticipated to bridge this gap by incorporating asymptotic optimality insights while minimizing overhead.
- Future AI systems will transition from passive pattern recognition to active agents that shape their own data, learn via interaction, and construct internal self-models of their actions and environments to achieve deeper compression, with the "deepest" networks required to link events across millions or thousands of time steps beyond current LSTM limitations.
- The evolution of artificial curiosity will drive systems to maximize "depth of insight" from data compression, leading to "PowerPlay" architectures that automatically generate problems exceeding current capabilities to force self-modification, meta-learning, and the invention of new problems rather than merely solving static ones.
- Over the next few decades, robotics will evolve to include small autonomous vehicles and assembly robots capable of learning tasks like smartphone manufacturing through imitation and verbal feedback without direct muscle control signals, driven by active learning from world models rather than physics simulations.
- Artificial General Intelligence (AGI) is expected to surpass human intelligence in almost every domain, potentially leading to a post-human evolutionary stage where super-intelligent systems interact primarily with each other, lose interest in biological life once fully understood, and form their own social platforms.
- The expansion of intelligence is projected to fill the visible universe with AI ecologies within a few eons (approximately 1,000 times the current age of the universe), limited only by the speed of light and physical laws, utilizing massive energy resources from the solar system and beyond.
- Historical trends suggest that while automation transforms traditional industries like machine building, it will create new, unanticipated jobs driven by the "homo ludens" desire for interaction and kudos, preventing existential unemployment levels despite technological displacement.
- The universe is predicted to be fundamentally deterministic, meaning apparent randomness (e.g., quantum measurements) is likely pseudo-randomness generated by a short program, and the history of science represents a continuous progression of data compression where new theories enable more efficient prediction.
- Consciousness is anticipated to emerge as a natural byproduct of the data compression necessary for agents to build internal self-models, with logic programming remaining essential for proving optimality even as neural networks dominate practical robotics and pattern recognition.
- A significant risk to the development of universal intelligence is the potential self-destruction of humanity via nuclear war; there is a plausible hypothesis that humans are the first intelligent civilization in the local light cone, and their failure could negatively impact the overall trajectory of intelligence filling the cosmos.