MIT 6.S094: Recurrent Neural Networks for Steering Through Time

Course Administrative Requirements:

• Students must create accounts on cars.mit.edu to receive credit.
• Code submissions for "DeepTrafficJS" and "DeepTeslaJS" are mandatory.
• The "DeepTraffic" challenge requires a neural network to drive faster than 65 miles per hour.
• A "deep thoughts" section has been added to user profiles for sharing experimental attempts and results.

Recurrent Neural Networks (RNNs) vs. Vanilla Networks:

• Vanilla Networks: Map fixed-size inputs to fixed outputs (e.g., ImageNet image-to-label); treat audio clips as fixed-size chunks.
• RNN Capabilities: Handle variable input and output sizes; process sequences where temporal dynamics matter more than spatial content (e.g., speech, natural language, video).
• Mapping Types:
  • One-to-many: Start a song; generate a sequence from a single input.
  • Many-to-one: Classify a speaker's gender from a sequence of speech.
  • Many-to-many: Machine translation (e.g., Spanish to English) where input and output sequence lengths differ.
• Core Mechanism: Outputs are looped back as inputs, creating a feedback loop that allows the network to process arbitrary sequence lengths with shared parameters across time steps.

Backpropagation and Optimization Fundamentals:

• Process: Computes gradients of the loss function with respect to parameters (weights/biases) by propagating error backward through the network using the chain rule.
• Training Goal: Minimize the loss function (objective function) by adjusting weights and biases in the opposite direction of the gradient.
• Key Components:
  • Gates: Operations decomposed into addition, multiplication, and max gates for gradient calculation.
  • Learning Rate: Determines the magnitude of weight updates; can be uniform or individual per weight.
  • Optimization Algorithms: Stochastic Gradient Descent (SGD) is standard; advanced variants like Adam address non-convex landscapes and saddle points.
• Challenges:
  • Vanishing Gradients: Gradients shrink to zero (often due to sigmoid saturation), preventing early layers from learning.
  • Exploding Gradients: Gradients grow excessively, causing instability.
  • Non-Convexity: Neural networks optimize highly non-linear functions where local minima do not guarantee global optimality.
  • Detection: Diagnosed via loss function stagnation, slow convergence, or "dead neurons" (e.g., ReLU units that never fire).

Long Short-Term Memory (LSTM) Architecture:

• Problem Solved: Vanilla RNNs struggle with long-term dependencies due to vanishing gradients over long sequences.
• Mechanism: Uses a "conveyor belt" state vector regulated by three gates:
  1. Forget Gate: Decides what information to discard from the previous state (via sigmoid).
  2. Input Gate: Decides what new information to store in the state.
  3. Output Gate: Controls what part of the state is outputted.
• Parameters: Weights and biases are shared across time steps, allowing the model to generalize to sequence lengths not seen during training.
• State Representation: A vector of activations (typically between 0 and 1 or -1 and 1) held in memory cells.

Key Applications of RNNs/LSTMs:

• Natural Language & Audio: Machine translation, character-level text generation, speech recognition, and voice synthesis (generating audio from text).
• Multimodal Tasks:
  • Visual Question Answering: Combining CNN image embeddings with LSTM text processing to answer questions about images.
  • Captioning: Generating syntactic sentences to describe images or video frames.
  • Visual Attention: Using RNNs to steer CNNs (sliding windows) to identify specific objects (e.g., house numbers).
  • Audio Synthesis: Generating sound (spectrograms) from silent video sequences (e.g., drumsticks hitting objects).
• Medical Diagnosis: Predicting conditions (diabetes, scoliosis) from sparse, variable-length electronic health record sequences.
• Financial Analysis: Predicting stock market movements using raw order book data and news article sequences.
• Autonomous Driving:
  • Baseline (DeepTesla): Single CNN image to steering angle (regression).
  • Advanced (Competition Winners): LSTMs process sequences of images (e.g., 10 frames) to predict sequences of steering angles, speed, and torque.
  • Transfer Learning: Pruning pre-trained CNNs (e.g., ResNet, VGG) to extract high-level features (e.g., 3,000 features) which feed into LSTMs for driving tasks.
  • Implementation Details: Competitive approaches often use sequence lengths of 10 to 50 and tune hyperparameters extensively.

Future Outlook and Limitations:

• Art vs. Science: Training often relies on manual hyperparameter tuning and intuition ("Stochastic Graduate Student Descent") rather than purely algorithmic solutions.
• Data Dependency: Success relies on massive datasets; transfer learning is a critical strategy to mitigate data scarcity.
• Scope: Most real-world data is time-series; RNNs/LSTMs are broadly applicable to any domain involving sequential dynamics.
• Upcoming Content: The lecture concluded by deferring "driver state" analysis to the next session and announced a White House guest speaker on AI policy.