Lecture, Tutorial
MIT 6.S094: Convolutional Neural Networks for End-to-End Learning of the Driving Task
- The course intends to continue discussing convolutional neural networks for autonomous driving and cover a case study on traffic light detection, contingent upon available time.
- Speakers plan to demonstrate the "DeepTesla" system live on a website towards the end of the lecture and provide a TensorFlow tutorial with code if scheduling permits.
- Students are expected to gain the ability to train neural networks for driving using real-world data in a browser via JavaScript, though larger, deeper networks utilizing GPUs will require offline TensorFlow usage.
- Computer vision is described as dominated by data-driven machine learning approaches, with humans noted as highly proficient at converting visual perception into semantics.
- Significant challenges in computer vision include viewpoint variation, object size differences, background clutter, inter-class variation, and illumination changes.
- The lecture will cover clustering images against semi-supervised and reinforcement learning, comparing k-nearest neighbors (38% accuracy) against state-of-the-art convolutional networks (95.4% accuracy) on the CIFAR-10 dataset, noting that human performance averages slightly above 94%.
- Convolutional neural networks are predicted to surpass human performance on CIFAR-10 and are expected to process spatial inputs like images and audio through shared parameters and pooling operations that reduce network size without learnable parameters.
- Network architectures are anticipated to map images to specific classes or other images for segmentation, with object detection achieved by segmenting scenes into candidates for classification.
- Statistical safety data indicates US drivers average 10,000 miles annually with 30,000 to 38,000 annual fatalities, whereas Tesla autopilot has accumulated 300 million miles (25% of total 1.3 billion) with one fatality compared to a human fatality rate of 1 in 90 million.
- MIT research involves instrumenting 17 Teslas to collect 5,000 hours and 70,000 miles of data, with forward roadway videos provided to the class containing encoded barcodes for stream synchronization.
- Sensor capabilities and limitations include LiDAR being ideal for 3D information but ineffective in rain or snow, regular C920 webcams used for onboard compression, radar providing reliable distance data, and software updates making the autopilot system increasingly conservative.
- Algorithms for scene understanding will leverage deep learning for object detection and segmentation, while SLAM remains a well-studied problem solvable via optimization techniques like SIFT, which deep learning has not yet surpassed.
- Future modules will utilize Recurrent Neural Networks for temporal video data and audio spectrograms to determine road conditions, alongside movement planning solved via optimization or reinforcement learning running tens to hundreds of millions of simulations.
- Upcoming topics include driver state detection focusing on head pose, eye gaze, and micro saccades, framed as a classification problem of six regions, and emotion detection via CNNs as a binary classification of frustrated versus satisfied states.
- End-to-end driving is conceptualized as a natural language generation problem where a network learns to navigate turns and intersections using a single forward roadway image input.
- The NVIDIA architecture utilizes a 66x200 pixel input image for a regression network outputting steering commands, while students are required to submit code and a car model for the class project.
- Neural networks are characterized as good memorizers requiring training data representation for every case, struggling to generalize from common cases to edge cases, and necessitating accuracy levels very close to 100%.
- The course will provide code in ConvNet.js and TensorFlow to train networks, visualize steering predictions against actual values, and cover building a network to detect traffic lights in a subsequent tutorial.
- Additional support for Docker or TensorFlow questions is scheduled for Friday from 5 to 7.