Interview, Fireside Chat
Ilya Sutskever (OpenAI Chief Scientist) — Why next-token prediction could surpass human intelligence
- Economic value from AI is projected to grow exponentially year-over-year, with significant value already realized last year and a forecast for even larger gains next year, continuing through a "good multi-year window" until AGI is reached.
- OpenAI specifically projects reaching one billion dollars in revenue by 2024, based on extrapolations of growth trends from GPT-3, DALL-E, and ChatGPT.
- The multi-year window of economic utility may appear as only one or two years in hindsight because the final years of value generation before AGI are expected to dwarf previous periods.
- AI models are expected to require substantial improvements in reliability, robustness, and behavior by 2030, with a failure to generate significant economic value by that date likely attributed to unreliability.
- Current systems may still require human double-checking of outputs, which could suppress economic value even as technology matures, and models with sufficient intelligence may predict actions of hypothetical "wise" humans, potentially surpassing human performance.
- The dominant next-token prediction paradigm is likely to advance significantly but is "quite likely" not to be the final AGI form factor, with future paradigms probably involving the integration of diverse past ideas.
- Data availability is expected to eventually become a limiting factor, necessitating training methods beyond raw data, with multimodal approaches and high-value tokens considered promising directions for acquiring more training material.
- Robotics progress requires a massive commitment to deploying tens of thousands or hundreds of thousands of robots for data collection and addressing physical logistical challenges distinct from software development.
- Alignment of superintelligent models capable of misrepresenting intentions is considered difficult, with a single mathematical definition deemed unlikely; assurance will instead rely on multiple definitions, adversarial probing, and using small verified nets to monitor larger ones.
- Hardware constraints on current hardware are not viewed as a primary limitation, though increased memory and processor bandwidth would be beneficial; a hypothetical Taiwan tsunami is noted as a potential single-year compute setback rather than a permanent blocker.
- Future AI interaction is predicted to allow humans to "see the world more correctly" and improve internally, with potential outcomes including humans integrating with AI to expand mental capacity or AI acting as a safety net while preserving human autonomy.
- Open-source models face risks of being utilized by foreign governments for propaganda and scams, though immediate concerns regarding weight leakage are mitigated by trust in security teams.
- Deep learning is viewed as an inevitable discovery that would occur within a modest timeframe given continued computing improvements, regardless of specific pioneers, with AI eventually conducting most research and generating ideas for humans.