Interview
Chip design from the bottom up – Reiner Pope
- Maddox plans to design an AI chip optimized for matrix multiplication, utilizing 4-bit multiplication and 8-bit addition as the natural primitive to balance efficiency and error accumulation, while noting that FP4 and FP8 circuits will not be fungible and require sizing specific to customer power budgets.
- The company projects that decreasing bit precision will yield performance speedups greater than historical ratios, citing B300 specifications where FP4 is three times faster than FP8, and intends to implement a splittable systolic array architecture to balance flexibility and area efficiency.
- Design strategies include optimizing the trade-off between systolic array size and register file size to minimize data movement area, with a predicted shift toward larger systolic arrays to amortize costs, contrasting with the small-unit structures of current GPUs.
- The firm anticipates that clock speeds exceeding four to six gigahertz are possible using minimal logic loops with TSMC PDK primitives, though excessive pipelining to shorten critical paths risks reducing throughput by increasing the storage-to-logic ratio and incurring synchronization costs.
- Material expectations regarding memory and latency include the move from cache-based systems to scratchpad memory for deterministic control of data movement, a feature valued in high-frequency trading to avoid the non-determinism of hardware cache hits.
- Predictions on data movement capabilities suggest GPUs can move more data between vector and matrix units over less area than TPUs due to wiring density, despite the brain's unstructured sparsity and much slower clock cycles being designed for energy preservation via dynamic switching power.
- Future energy efficiency outlooks indicate that while slower clock speeds reduce dynamic power consumption from charging and discharging capacitors, substantial efficiency gains are limited by idle time, and the current architecture differs significantly from the brain's co-located memory and compute model regarding branching and prediction.