Jun 27, 2020, 3:21 PM ETInterview

David Patterson: Computer Architecture and Data Storage | Lex Fridman Podcast #104

Microprocessors, invented roughly 40 years ago, enabled computers to shrink from room-sized machines to devices fitting in cell phones.
Computing speed has increased by a factor of one million over the last 50 years, while costs have dropped significantly.
There are now likely more microprocessors than humans on Earth, with a ratio potentially exceeding 10:1.
Moore's Law, formulated by Gordon Moore in 1965, predicted the number of transistors on a chip would double every two years; this held true for approximately 50 years but has since slowed.
The shift from mainframes to ubiquitous personal computing was a "shocking" societal event that moved computing from a niche industry into mainstream public awareness.

Modern computers operate on five fundamental components: input, output, memory, arithmetic unit (brawn), and control unit (brain).
Layered abstraction allows engineers to manage complexity by "suspending disbelief" and interacting with higher-level interfaces without needing to understand the underlying transistor-level details.
There are typically 10 to 20 layers of abstraction in modern computing, ranging from high-level application software down to the hardware transistor.
The concept of "open source" is expanding from software to hardware, allowing researchers and developers to access and modify the design of processors and instruction sets.

In the 1980s, a controversial shift occurred from Complex Instruction Set Computing (CISC) to Reduced Instruction Set Computing (RISC), championed by Berkeley, Stanford, and IBM.
CISC utilized powerful, multi-step instructions that were difficult for compilers to optimize and required complex hardware.
RISC proposed using a smaller set of simple, single-step instructions (like add, subtract, load, store) that could execute much faster.
RISC architectures won the debate because, despite requiring 50% more instructions to perform a task, they executed those instructions four times faster due to simpler hardware.
Intel successfully bridged the gap by translating CISC (x86) instructions into RISC-like micro-operations within their hardware, maintaining compatibility while leveraging RISC efficiency.

RISC-V, developed at Berkeley around 2010, is an open-source instruction set architecture (ISA) designed to avoid the proprietary limitations of x86 (dominated by Intel) and ARM (dominated by ARM Holdings).
The RISC-V foundation grew from 50 attendees at its first conference in 2015 to 1,700 attendees in the most recent gathering.
RISC-V features a "core" set of approximately 40 essential instructions that all software can run on, with optional subsets added for specific performance needs without breaking compatibility.
This "slim core" design allows for easier education, faster iteration, and adaptation to new computing mediums (e.g., biological computing) compared to bloated proprietary ISAs.
RISC-V is currently seeing its most significant adoption in the Internet of Things (IoT) and embedded devices, where no single proprietary vendor dominates the market.

Moore's Law has slowed, with general-purpose processor performance now improving by only a few percent annually rather than doubling every 18 months.
Hardware designers are shifting toward domain-specific accelerators (e.g., TPUs, NPUs) that optimize performance for specific workloads rather than general computation.
The Machine Learning (ML) revolution coincides with this shift; ML relies heavily on matrix multiplication, a task where specialized accelerators can deliver massive speedups.
The "Software 2.0" paradigm describes a shift where programs are defined by data and learned from examples rather than explicitly coded by humans.
Intel acquired Habana Labs for $2 billion to integrate its high-performing ML accelerators, replacing its previous internal ML hardware efforts (Nervana) which lacked sufficient software support.

The MLPerf benchmark suite was created to provide fair, transparent metrics for comparing machine learning hardware, similar to how SPEC benchmarks standardized RISC architecture comparisons in the 1980s.
MLPerf now involves 50 companies and 10 universities, ensuring that performance claims are verified against a common set of tasks rather than marketing assertions.
David Patterson notes that without benchmarks, fields like computer vision previously lacked objective measures of progress, leading to subjective "hunches" rather than quantitative advancement.

RAID (Redundant Arrays of Inexpensive Disks), co-authored by Patterson in 1988, introduced the concept of using multiple cheap hard drives to create a system that is faster, cheaper, and more reliable than a single large, expensive drive.
The "R" in RAID stands for redundancy, which allows the array to survive multiple disk failures without data loss.
The storage industry has shifted from magnetic hard disk drives (HDDs) to semiconductor flash memory, particularly for mobile and laptop devices, though HDDs remain cost-effective for cloud data centers.
Despite the technology shift, the logical principles of RAID (data striping, mirroring, and parity) remain applicable to flash storage.

Patterson argues that teaching and research are synergistic; explaining complex concepts to students forces researchers to deepen their own understanding and question assumptions.
His competitive wrestling career at UCLA and El Camino Community College instilled self-confidence, team-building skills, and a willingness to "stand up" against falsehoods in both physical and intellectual arenas.
Patterson measures a "life well-lived" not by financial wealth or publication counts, but by the depth of relationships and the positive influence on others' lives.
He attributes his 50-year marriage to the practice of acknowledging mistakes and expressing love, summarized by the phrase: "I was wrong. You were right. I love you."

Quantum computing is not expected to be commercially viable or error-corrected until around 2030, requiring extreme conditions (near absolute zero) and centralized data centers rather than consumer devices.
Moore's Law has technically ended regarding transistor density doubling; while industry marketing may claim otherwise, quantitative data shows a clear slowdown in transistor scaling.
The immediate future of computing relies on software optimization and domain-specific hardware accelerators rather than waiting for general-purpose speed increases or quantum breakthroughs.