Lecture
MIT 6.S094: Introduction to Deep Learning and Self-Driving Cars
Lex FridmanLex Friedman, Dan Brown, William Angio, Spencer Dodd, Benedict Jenick, Andrej Karpathy, Hans Moraveck
- The course introduces deep learning methods for autonomous driving via two projects, "Deep Traffic" and "DeepTesla," culminating in a competition where students design neural networks to exceed a 65 mph speed limit for a special prize; "Deep Traffic" trains in minutes via a browser using ComNetJS, while "DeepTesla" maps Tesla vehicle images to steering angles to match or exceed human performance.
- Educational resources including lecture videos, code tutorials, slides, and a FAQ website are hosted at deepcars.mit.edu and cars.mit, with lecture videos becoming available a few days post-delivery.
- The curriculum covers deep reinforcement learning, convolutional and recurrent neural networks, and their application to perception, localization, mapping, control planning, and driver state detection, noting that while the brain has 100 to 1,000 trillion synapses, current artificial networks have only 10 billion connections.
- Deep learning is identified as efficient for formal tasks like board games but faces challenges with mundane tasks like speech and object manipulation due to high data requirements, the difficulty of defining reward functions, and the risk of AI winters if hype outpaces capability.
- Technical limitations include the vulnerability of neural networks to noise and spoofing attacks, such as LiDAR replay attacks, and the high cost of annotating ground truth data, contrasting with human efficiency in learning from single examples.
- Future developments aim to optimize deep learning for mobile devices, expand unsupervised and reinforcement learning, and explore multimodal learning, with a five-year vision targeting smaller, cheaper hardware and better video summarization or generation.
- Current research focuses on semi-autonomous vehicle handovers, internal driver monitoring, and the use of sensors like radar, lidar, and cameras, acknowledging that autonomous driving requires a margin of error of one fatality per 100 million miles, whereas the Google team recorded 341 disengagements in San Francisco between 2014 and 2015.
- The distinction between special purpose AI, which achieves well-defined goals in constrained environments, and general AI, which handles unconstrained goals, remains an open inquiry, as does the question of whether reasoning requires larger networks or entirely new mechanisms.