Summary of Arjun Guha's Talk on LLMs, Programming, and Interpretability

Benchmarking and Low-Resource Languages

• Current Benchmark Saturation: Major coding benchmarks (HumanEval, MBPP) are approaching saturation, with top models achieving ~90% pass rates on Python, though ~10% of problems contain faults.
• Multilingual Limitations: Despite training on multilingual data, models historically performed significantly worse on low-resource languages (e.g., OCaml) compared to Python in early evaluations (e.g., OpenAI Codex scored ~45% on Python vs. much lower on other languages).
• Multi-PLE Benchmark: Researchers developed the "multi-PLE" benchmark, the first large-scale multi-language evaluation using trivial transpilers to convert Python benchmarks into languages like Rust and OCaml.
• Language-Agnostic Benchmarks: New methods ("Agnostics") use capable LLMs to transform Python benchmarks into language-agnostic problems with explicit input/output formatting, enabling easier evaluation across diverse languages like Fortran and OCaml.
• Performance Gaps: On language-agnostic HumanEval tasks, a base model trained for OCaml achieved 17% success vs. 10% for the base, while frontier models (e.g., GPT-5 Mini) achieved 72%.

Internal Model Mechanisms (Mechanistic Interpretability)

• Activation Steering: The speaker utilized activation steering (adding vector offsets derived from difference-of-means) to manipulate model behavior without retraining.
• Language Directionality: PCA projections of intermediate activations show that models consistently separate Python and OCaml prompt processing across transformer layers, creating a "language vector" that can be used to switch target languages.
• Correcting Failure Modes: Steering vectors computed by contrasting correct vs. incorrect solution trajectories can improve model performance; however, this primarily aligns the model rather than endowing it with new knowledge.
• Type Prediction Insights: Experiments on Python and TypeScript revealed that activation steering effectively corrects type prediction errors (preventing type errors) but does not improve type precision (filling in the exact original annotation).
• Shared Type Representations: Steering vectors trained on Python type errors were equally effective when applied to TypeScript tasks, suggesting a shared internal representation of types across languages.
• Misprediction Analysis: Models often fail type inference based on variable names (e.g., predicting int for a variable named n used as a string) rather than structural analysis, a failure mode that can be mitigated via steering.

Human-Model Interaction and Education

• Student Experiment Design: A study with 120 CS1 students (Python-only background) in early 2023 tested their ability to prompt an OpenAI Codex model to solve six programming tasks within 60 minutes.
• Success Metrics: "Eventual success rate" (allowing infinite retries) was high, but the success rate counting every failed attempt was significantly lower, highlighting the difficulty of effective prompting.
• Prompt Trajectory Data: The study produced a public dataset of 2,000+ prompts containing full interaction trajectories, unique for capturing the evolution of prompts from failure to success or abandonment.
• "Clue" Framework: Analysis of prompts revealed that success depends on including a specific set of essential "clues" (facts); if even one critical clue is missing, success probability drops sharply.
• Student Mental Models: Introductory students often fail to include necessary clues (e.g., rounding to two decimal places) due to a lack of understanding of model priors and floating-point behavior, leading to iterative cycles of ineffective edits.
• Prompt Engineering Fallacy: Students often believe they need to fix "vocabulary" or syntax; experiments showed that synonym substitution has no impact on model performance, whereas missing semantic "clues" is the primary failure point.
• Educational Implications: There is a tension between using LLMs for productivity and the educational goal of learning; LLMs may "short-circuit" the learning process if students rely on them to complete assignments without understanding the underlying mechanics.

Industry Trends and Future Directions

• AI Coding Agents: The adoption of AI coding agents (e.g., Cursor, Cloud Code) has surged, with ~1.3 million commits co-authored by agents detected on GitHub within the first four months of release.
• Productivity Debate: While some claim LLMs double productivity, controlled studies (e.g., recent meta-studies) suggest LLMs may actually slow down developers in specific contexts.
• Future Research: The speaker emphasizes moving beyond "can the model do task X" benchmarks to understanding human mental models of LLM capabilities and utilizing the formal properties of code (type checkers, test runners) to probe model internals.