Conference Presentation, Fireside Chat, Interview
Fei-Fei Li: Spatial Intelligence is the Next Frontier in AI
- Dr. Fei-Fei Li defines her career mission as solving "hard, bordering delusional" problems, specifically asserting that Artificial General Intelligence (AGI) cannot be achieved without spatial intelligence.
- Li views Spatial Intelligence as the ability to understand, generate, reason about, and interact within the 3D world, a capability that evolved in animals over 540 million years, contrasting with language which emerged in less than 500,000 years.
- World Labs, the company Li founded and currently leads as CEO, is dedicated to building "world models" that capture 3D structure and spatial intelligence beyond flat pixels or purely linguistic representations.
- The founding team at World Labs includes three world-class technologists: Justin Johnson, Ben Mildenhall (author of the NeRF paper), and Christoph Lassner (creator of the Pulsar model, the seed of Gaussian Splatting).
- Li characterizes the problem of spatial intelligence as significantly harder than Language Learning Models (LLMs) due to the 3D nature of reality, the mathematical ill-posedness of projecting 3D to 2D, and the dual challenge of reconstruction versus generation in a world governed by physics.
- Unlike LLMs which process purely generative 1D signals (language), spatial AI must handle the combinatorial complexity of 3D structures, sensory projections, and the fluid interplay between generating virtual worlds and reconstructing real-world physics.
- ImageNet, conceived around 2007 and released in 2009, was the first project to pioneer data-driven machine learning by downloading a billion images from the internet to create the world's visual taxonomy, addressing the lack of data that previously hindered algorithm generalization.
- The pivotal moment for modern AI occurred in 2012 when the AlexNet team, led by Alex Krizhevsky and Jeff Hinton, achieved a major step change in the ImageNet Challenge using Convolutional Neural Networks (CNNs) on two GPUs.
- In 2015, Li and her student Andrew Karpathy published pioneering work on image captioning, enabling computers to "tell the story" of a scene, a long-held dream of Li's that marked the collision of natural language and visual intelligence.
- Li notes that the trajectory of computer vision has moved from object recognition (solving for objects) to scene understanding (storytelling) and is now advancing toward full world modeling and generation via diffusion models.
- Li identifies "intellectual fearlessness" as the single most critical trait for hiring and research success, defined as the courage to embrace hard problems and go "all in" regardless of background or resources.
- Li's hiring criteria for World Labs prioritizes engineering, product, 3D, and generative model talents who possess the passion and fearlessness to solve spatial intelligence problems.
- The team at World Labs is currently adopting a hybrid approach to data, combining large-scale data acquisition with a strict focus on data quality to address the scarcity of accessible spatial data on the internet.
- Li advises PhD students to target fundamental, interdisciplinary "North Star" problems in academia—such as scientific discovery, AI theory, causality, and small data learning—that are not easily solvable by industry using brute-force compute and data.
- Li advocates for a non-dogmatic approach to open source, arguing that the ecosystem is healthiest when companies choose between open, closed, or tiered strategies based on their specific business models, while ensuring open source efforts in both public and private sectors are protected.
- Li believes AGI is likely a unified system rather than a multi-agent framework, citing the biological brain as a monolithic structure with specialized functional areas (visual, language, motor cortices) that operate as a whole.
- Li describes her transition from academia to entrepreneurship as a comfort zone, emphasizing the value of "ground zero" thinking where one ignores past achievements or external opinions to build something new.
- The concept of the Metaverse is identified by Li as a primary application area for spatial intelligence, driven by the convergence of hardware capabilities and the need for robust 3D content creation via world models.
- Li's personal history includes immigrating to the U.S. as a teenager, running a dry-cleaning business (laundromat) for seven years to fund her physics degree at Princeton, and later founding the Human-Centered AI Institute at Stanford as a university startup.
- Li emphasizes that graduate school is the appropriate venue for those driven by "burning curiosity," whereas startups must balance curiosity with commercial viability and investor expectations.
- Li's work on ImageNet resulted in over 80,000 citations and effectively kicked off the "data problem" leg of AI development, shifting the field from algorithm-centric to data-centric methodologies.
- Li warns against the "garbage in, garbage out" principle regarding data quality, asserting that high-quality data is essential for training effective spatial world models despite the current scarcity.