Nov 29, 2023, 10:49 AM ETInterview, Podcast

Working at top AI labs without an undergrad degree | Chris Olah

Chris Olah dropped out of university in 2009 to support a friend falsely accused of terrorism charges in Toronto; the friend was acquitted, but the experience soured Olah's view of national security agencies.
Olah received a $100,000 Thiel Fellowship in 2012, a scholarship for individuals under 20 to pursue entrepreneurship or research instead of university.
He pivoted from 3D printing and open-source CAD software to machine learning research after exposure to deep learning concepts via Michael Nielsen's seminars and a meeting with Holden Karnofsky of GiveWell.
Olah lacks a formal degree but secured positions at Google Brain (2013–2018) and OpenAI (2018–2022) through cold emailing, networking, and demonstrating value via research and visualization rather than credentials.
He spent two years as an intern at Google Brain before becoming a full-time researcher, a tenure that allowed him to work on visualizing neural networks and developing feature visualization tools.

Olah is a second author on the 2015 "Deep Dream" paper, which generated widely publicized hallucinogenic images by maximizing neuron activations in convolutional neural networks.
He founded the Clarity research team at OpenAI to focus on neural network interpretability, specifically pursuing the "circuits agenda" to fully understand how networks implement algorithms.
In late 2022, Olah left OpenAI with colleagues to establish a new AI lab focused on large models and safety.
He co-authored the influential paper "Concrete Problems in AI Safety" and contributed to the development of TensorFlow.
Olah founded the academic journal Distill in 2018 to publish interactive, visual explanations of technical concepts, though he later concluded that career incentives were not the primary barrier to high-quality exposition; rather, the high effort required is the main deterrent.

Olah advises that for most people, attending university is still the optimal path; non-traditional routes require exceptional self-discipline and the ability to thrive without external forcing functions.
He identifies the lack of a degree as a significant hurdle for US immigration, noting that almost all visas require an undergraduate degree, with the exception of the "Alien of Extraordinary Ability" visa.
Olah cites a lack of access to the social networks and peer groups typically found on university campuses as a major, underestimated personal cost of not attending college.
In the machine learning field, formal credentials are less critical than proven research output, as evidenced by Olah's hiring and the fact that some top researchers lack PhDs.
Computer science differs from other fields by prioritizing conference papers and preprints (e.g., arXiv) over traditional peer-reviewed journals, a norm that allows for more flexible career entry.

Olah advocates for developing skills at the "Pareto frontier" of intersections (e.g., machine learning + artistic illustration) to create unique value propositions that face less competition than mastering a single skill.
He recommends "pair programming" as the highest-leverage method for transmitting technical technique, as it allows for the observation of hidden workflows that are difficult to convey verbally or in writing.
Olah suggests viewing research as a market where one can either make a "popular" field slightly more efficient or "beat the market" by working on neglected but promising ideas.
He emphasizes that cold emailing is effective only if it demonstrates deep familiarity with the recipient's work, ideally involving the reading of multiple papers and specific, thoughtful questions.
Olah argues that good explanations are vital to prevent "research debt," where the accumulation of ideas makes the field inaccessible to new researchers, effectively slowing the growth of the scientific community.

Olah introduces the concept of "micro-events" (e.g., micro-marriages, micro-friendships) to quantify low-probability, high-impact life events, allowing individuals to track progress toward significant outcomes like finding a partner or a career-defining job.
He notes that explanations often fail because authors possess resources readers lack, specifically a full set of memorized terms and pre-existing motivation, leading authors to overload the reader's working memory.
Olah suggests that improving explanations yields nonlinear returns; a single superior explanation can become the default reference for a field, saving millions of readers significant time.
He believes that the "technical debt" in science is comparable to software engineering, where rapid progress leads to messy, poorly documented ideas that hinder future innovation.