Karl Iagnemma & Oscar Beijbom (Aptiv Autonomous Mobility) - MIT Self-Driving Cars

Aptiv & Autonomous Vehicle Deployment Overview

• Carl Bayboom serves as President of Aptiv Autonomous Mobility, having previously founded Neutonomy, which was acquired by Aptiv (formerly Delphi Technologies, spun from GM) in 2017.
• Aptiv is a Tier 1 supplier with 156,000 employees, $13 billion in revenue, and operations in 50 countries, focusing on industrializing software/hardware for high-mileage reliability.
• The Aptiv autonomous driving team consists of approximately 700 employees operating a fleet of ~120 vehicles across the US (Pittsburgh, Boston, Vegas), Singapore, and China.
• In Las Vegas, Aptiv partners with Lyft to provide public rides; the fleet includes 30 active autonomous vehicles operating 20 hours/day, 7 days/week.
  • The fleet has completed over 30,000 rides with 50,000 passengers.
  • Total miles driven in Vegas exceed 1 million.
  • Customer satisfaction star rating is 4.95/5.0.
• Operational environments vary significantly, including Singapore's left-hand traffic, heavy construction, and unpredictable pedestrian behavior, versus Las Vegas's warm weather and mid-density urban conditions.
• The company utilizes a "rule book" architecture for decision-making to allow flexible reprioritization of traffic rules for different jurisdictions without manual code rewriting for left/right-hand driving.

Technical Philosophy: Safety, Validation, and Neural Networks

• The industry has moved away from the "pixels-to-actuator" single black-box neural network approach due to difficulties in proving safety and validating the system.
• Safety strategy relies on two dimensions: technical safety (verifiable, provable performance) and perceived safety (user confidence and comfort).
• Carl Bayboom advocates for "caging" neural networks: embedding deep learning models within a rigorous, verifiable safety architecture to handle end-to-end autonomy claims.
• Three primary validation challenges are identified for deep learning in safety-critical systems:
  • Data Trust: Ensuring data sufficiency, coverage, and high-quality third-party annotation to avoid "garbage in, garbage out."
  • Algorithm Trust: Verifying invariance, stability against perturbations (e.g., adversarial examples), and interpretability for regulatory explanation.
  • Continuous Validation: Addressing the cost and complexity of revalidating code after minor changes and when expanding to new geographies (continuous deployment).
• RAND Corporation analysis suggests a fleet must drive 275 million miles without a crash to statistically prove a lower fatality rate than a human (95% confidence).
• Current industry progress includes ~12-14 million autonomous miles driven across hundreds of code builds and environments, necessitating a hybrid approach of on-road testing, simulation, and regression testing.

Machine Learning & Perception Architecture (Oscar Bayboom)

• Oscar Bayboom presented PointPillars, a novel LiDAR point cloud encoder that processes point clouds in real-time, outperforming previous methods on the KITTI benchmark while running at ~60 Hz.
  • Methodology: Instead of converting points to 3D voxels (as in VoxelNet), PointPillars groups points into vertical columns (pillars) and learns features directly, feeding a pseudo-image tensor into a standard 2D CNN backbone.
  • Performance: Achieves a 10x to 100x speedup over VoxelNet (reducing encoder runtime to 1.3 ms) and achieves state-of-the-art detection performance for cars and cyclists.
  • Architecture: The system is fully parameterized and differentiable, allowing end-to-end training.
• The team released nuScenes, a new open benchmark dataset containing 1,000 scenes (20 seconds each) collected from a 360-degree sensor suite.
  • Dataset Features: Includes 1 million+ 3D bounding box annotations, synchronized data from cameras, LiDAR, and radar, and a full world-coordinate system.
  • Goal: To advance 3D perception research by providing a more comprehensive benchmark than KITTI, which lacks radar and full 360-degree coverage.
  • Availability: A teaser release of 100 scenes was made available immediately; the full dataset release was scheduled for approximately one month post-talk.

Q&A and Strategic Considerations

• Data Generalization: Oscar Bayoom clarified that nuScenes is an academic research tool for algorithm development, not a standalone safety validation dataset sufficient for regulatory approval.
• Sensor Agnosticism: The PointPillars method preserves Z-axis information within learned features, avoiding the information loss of early voxel-based stratification.
• 5G Connectivity: The company views 5G as a "nice to have" for fleet coordination and remote teleoperation (high bandwidth/low latency) but does not rely on it for core vehicle safety, as infrastructure deployment is outside their control.
• Regulatory Oversight: Carl Bayoom noted the ambiguity in future regulation, stating the automotive industry might adopt external third-party certification (like nuclear or aerospace) or continue self-certification, with the US Department of Transportation recently issuing draft guidelines.
• Environmental Adaptation:
  • LiDAR data is more amenable to augmentation than visual data; synthetic objects (e.g., a piano on the road) can be added to point clouds by inserting 3D models and simulating returns.
  • Training data must explicitly support deployment conditions (e.g., snow); the model does not inherently generalize to unseen weather without specific training or augmentation.
• Cross-City Transferability: The decision-making engine is designed to handle diverse global norms (e.g., Singapore vs. Vegas) via rule weighting rather than hard-coded logic, though sensor performance and environmental factors (rain, construction, jaywalking) still require specific tuning and validation.