Aug 14, 2020, 6:53 PM ETInterview

Dileep George: Brain-Inspired AI | Lex Fridman Podcast #115

Engineering vs. Simulation: Building a functional brain-inspired AI requires a theoretical understanding of computational principles, not just simulating biological details (as attempted by the Blue Brain Project); simulation without theory is un-debuggable.
Iterative Loop: The optimal path to understanding the brain is to build functional models using neuroscience insights, verify their performance, and use the model's failures/successes to inform new biological experiments.
Biological Plausibility: Learning algorithms do not strictly need to be biologically plausible at the molecular level; the focus should be on the computational logic of credit assignment and inference, allowing for engineering shortcuts that biology does not use.
Consciousness: The name "Vicarious" reflects the idea that consciousness arises when a system's modeling apparatus turns inward to model its own existence, though mortality and death are not fundamental requirements for artificial intelligence.
Mortality & Motivation: Unlike humans, AI systems do not fear death because they can be copied; therefore, the urgency driving human goal generation via mortality is not a prerequisite for AGI.

Feedback Prevalence: The brain contains significantly more feedback connections than feed-forward ones; the visual system actively projects world models onto sensory input rather than passively processing data.
Inference as Explanation: Perception is an iterative inference process where the brain seeks the "best explanation" for sensory evidence, combining top-down predictions with bottom-up data.
Explaining Away: The brain employs probabilistic "explaining away" mechanisms where competing hypotheses (e.g., "burglar" vs. "earthquake") compete, and evidence for one suppresses the other.
Cortical Microcircuits: A cortical column functions as a binary variable (e.g., "is an edge present?") storing a concept, while connections between columns encode relationships between variables.
Temporal Dynamics: Experiments show that in visual tasks, the brain resolves contours (edges) before filling in surfaces, and illusory edges generated by context appear later than real edges, driven by feedback.
Natural Signal Priors: Human vision is optimized for "natural signals" (continuous, structured patterns) rather than artificial patterns like QR codes, implying AI models should encode priors derived from the statistical properties of the natural world.

Joint Modeling: Unlike standard deep learning that separates perception and cognition, RCN treats perception, cognition, and language as interconnected, allowing for top-down controllability and imagination.
Generative & Controllable: The model is generative but not photorealistic; it factors the world into distinct entities (foreground/background, object parts, texture/shape) allowing users to manipulate and simulate scenarios (e.g., changing the color of the "E" in the Google logo).
Lateral Connections: The architecture includes lateral connections to enforce spatial compatibility and constraints, preventing uncoordinated or jagged outputs during generation.
Dynamic Inference: The model performs dynamic inference on the fly rather than relying on amortized inference (brute-force training of all possibilities), allowing it to handle novel configurations without prior examples.
Non-Backpropagation Training: RCN does not use backpropagation; it relies on local learning rules and iterative inference loops, diverging from standard convolutional neural network (CNN) training methods.
Interpretability: The system provides a full "explanation" of its output, mapping specific input pixels to hallucinated contours and surfaces, making errors similar to human perceptual failures.

CAPTCHA Success: RCN successfully cracked text-based CAPTCHAs (prevalent in 2014-2015) that remain unsolvable by current deep learning systems without specific training on that style.
Few-Shot Learning: The model achieves 95% accuracy on tasks like MNIST with only tens or hundreds of training examples, vastly outperforming systems requiring massive datasets.
Generalization: The architecture excels at out-of-distribution generalization because it learns the underlying structural priors of the world (causality, object permanence) rather than memorizing statistical correlations in large datasets.
Human vs. Machine: A human child can solve a new CAPTCHA style instantly, whereas AI systems currently require specific training data for each new font or distortion type, highlighting the gap in "common sense" reasoning.
Scientific Publication: The work was published in Science, addressing feedback connections, recursive inference, and lateral connections as a coherent framework for vision.

Limitations of Scaling: Scaling up language models (GPT-3/4) creates a "model of text" rather than a "model of the world," failing to capture physics, causality, or interactivity necessary for true reasoning.
Feed-Forward Deficit: Transformer architectures are fundamentally feed-forward and lack the recursive feedback loops required for counterfactual reasoning, interventions, and simulation.
Lack of World State: These models generate coherent text but cannot maintain a persistent world state or simulate scenarios to verify truth, leading to logical inconsistencies in long-range reasoning.
Information Fidelity: Text is a lower-fidelity representation of reality than physical interaction; compressing text does not inherently generate the rich, grounded knowledge required for AGI.

Pre-Verbal Concepts: The research focuses on learning concepts from raw visual and motor interaction (pre-language) before integrating language, positing that grounded simulation is a prerequisite for understanding.
Language as Simulation Control: Language is viewed as a query mechanism to control the internal world simulation, allowing the system to answer questions by running scenarios in the visual/motor cortex rather than retrieving text associations.
Episodic Memory: A proposed architecture separates statistical world models (cortex) from episodic timelines (hippocampus), where the hippocampus acts as a pointer system to replay specific experiences and index them against the learned world model.

Neuralink Feasibility: BCI is considered a promising field for therapeutic applications (prosthetics, quadriplegia), though high-bandwidth communication faces immense challenges regarding brain plasticity, safety, and long-term biological adaptation.
Adaptation Strategy: Successful BCI may rely on the brain adapting to the interface (neuroplasticity) rather than the machine trying to decode the brain's natural language perfectly.
Career Advice: Aspiring researchers aiming to build brain-inspired AI are advised to pursue Computer Science, Electrical Engineering, or Machine Learning while self-studying neuroscience, as these disciplines provide the necessary tools for engineering intelligence.
Recommended Reading: Key texts include Judea Pearl's Probabilistic Reasoning in Intelligent Systems (causality), Causality by Judea Pearl, and Bishop's Boys regarding the history of flight and engineering analogies.