Chelsea Finn: This is the State of the Art in Robotics

• Organizational Focus: Physical Intelligence, founded two years ago, aims to develop general-purpose physical AI models capable of performing any task in unstructured real-world environments.
• Performance Milestones: Recent capabilities demonstrated include washing greasy pans, peeling carrots, making grilled cheese sandwiches, and slicing zucchini, moving beyond earlier demonstrations of laundry folding.
• Reliability Thresholds: The organization targets >90% success rates for complex manipulation tasks (e.g., espresso making) to ensure utility in real-world deployment without human supervision.
• Autonomy Requirements: Unlike software AI where human-in-the-loop correction is acceptable, physical AI requires near-perfect autonomy because errors directly impact the physical world.
• Industry Precedent: Waymo's achievement of 250,000 weekly autonomous rides serves as validation that machine learning systems can operate trustworthily in the physical world.
• Efficiency Bottleneck: Standard reinforcement learning (RL) in robotics is computationally expensive; one million one-minute trajectories would require approximately 700 robot days to achieve high reliability via brute-force scaling.
• RL Optimization Strategies:
  • Early Intervention: Human teleoperation intervenes to correct "dead-end" trajectories (e.g., separating two cardboard boxes) rather than letting the robot waste time attempting impossible paths.
  • General Value Functions: A unified value function estimates progress across diverse tasks (e.g., folding shirts vs. retrieving items from a fridge) to amortize the cost of trial-and-error learning.
• Throughput Gains: Applying these RL improvements resulted in a 2x increase in task throughput (success rate combined with speed) compared to pre-trained or supervised fine-tuned baselines.
• Memory Architecture:
  • Multi-Timescale Memory: To handle long-horizon tasks (10–15 minutes) without overwhelming context limits, the system uses short-term video memory (10 seconds) and compressed textual summaries for long-term history.
  • Task Enablement: This architecture enables non-repetitive, multi-step workflows such as full kitchen cleaning (wiping, drying, organizing, washing) without constant re-prompting.
• PI-07 Model Capabilities:
  • Single Model Approach: The PI-07 foundation model achieves performance matching or exceeding fine-tuned specialists in tasks like shirt folding, precision assembly, and trash bag replacement.
  • Zero-Shot Generalization: The model successfully interacted with an air fryer despite having virtually no training data on that specific appliance.
  • Embodiment Generalization: The model generalized the task of shirt folding from its training platform (manipulator) to a new industrial platform (BiARM UR5e) without any task-specific training data on the new robot.
• Data & Prompting Impact:
  • Diversity Necessity: Removing the most diverse subset of training data caused a dramatic drop in performance on held-out tasks, highlighting the importance of heterogeneous datasets.
  • Metadata Prompting: Including metadata prompts (episode length, data quality) allowed the model to leverage low-quality data, whereas models without this prompting degraded when exposed to such data.
• Deployment Ecosystem: Models are currently being deployed by YC-backed companies (Ultra, Weave) for real-world tasks like laundry folding and warehouse packaging, and adapted for diverse embodiments including drones, surgical robots, and tractors.
• Future Scaling: The technology has transitioned from a "pre-trained" regime (2014-era AI) to an "out-of-the-box" generalist regime (2022-era AI), similar to the shift from BERT to GPT in language models.

Q&A Substantive Insights

• Adoption Timeline: Physical AI adoption will likely be slower than software AI (e.g., ChatGPT) due to the logistical constraints of deploying physical hardware, though capability parity with ChatGPT is expected within a few years.
• Strategy for Small Teams: Small teams should immediately adopt generalist policies (e.g., PI-05) and fine-tune them, rather than scaling per-site models, except in highly constrained offline environments.
• Data Generation: The robotics equivalent of internet-scale text data is autonomous robot experience (self-play), which is critical because watching human video data does not transfer to physical dexterity.
• Control Architecture: The models output target joint positions or 3D gripper coordinates, which are then tracked by downstream PD controllers; direct torque/voltage control is possible but currently less proven as a bottleneck.
• Speed Improvements: Recent "RL token" releases have demonstrated policies faster than human teleoperation; future speed gains will come from either generating faster training data or training models that exceed human teleop speed.
• Emergent Capabilities: The PI-07 model demonstrated emergent symmetry (equivalence) by successfully assembling a pinwheel using opposite hands than those seen in training data, indicating learned equivariance.