Conference Presentation, Fireside Chat, Interview
Building the Real-World Infrastructure for AI, with Google, Cisco & a16z
- The current infrastructure build-out is projected to be 100 times the scale of the late 1990s internet expansion, a cycle with no prior equivalents and current projections that grossly underestimate future needs.
- Supply chain constraints and cash availability are expected to persist for a period of three to five years, creating a sustained period of scarcity in power, compute, and networking resources.
- Space and power depreciation cycles are estimated to span 25 to 40 years, leading to a long-term strategy where data centers are constructed where power is available rather than moving power to facilities.
- Networking infrastructure must transform to support scale-up and scale-out requirements, enabling new architectures where data centers 800 to 900 kilometers apart function as a single logical unit to handle immense, bursty bandwidth demands.
- The computing stack is predicted to become unrecognizable within five years due to a co-design process integrating hardware and software, driving an industry evolution toward "one company" models with deep, months-long design partnerships.
- A "golden age of specialization" is anticipated for processors, with TPUs expected to be 10 to 100 times more efficient per watt than CPUs for specific tasks, while the cycle from concept to production must be shortened from the current 2.5 years.
- Geopolitical structures will influence architectural choices over the next three years, with potential divergences such as China utilizing 7nm chips with abundant power versus the US utilizing 2nm chips with different thermal and efficiency profiles.
- Inference-native infrastructure is expected to evolve separately from training infrastructure, focusing on latency and memory optimization, while reinforcement learning on the critical path of serving will gain importance.
- To avoid monopolistic control, specifically over Broadcom-based systems, the industry must prioritize silicon diversity and maintain an open ecosystem at every layer of the stack.
- AI tools are expected to drive a 3x productivity increase within the next year through instruction set migration and codebase agnosticism, with code migration, debugging, and front-end development showing particularly strong results.
- AI agents and frameworks are predicted to become transformative over the coming 12 months, enabling intelligent routing layers and dynamic optimization of the software development lifecycle, while businesses relying on "thin wrappers" face short lifespans.
- The "intelligence per dollar" metric will remain a perpetual business driver, creating a self-fulfilling prophecy where performance demands for inference remain high despite cost reductions.
- Significant innovation is expected from Cisco in silicon, networking, security, and applications over the next 12 months, including transformative shifts in how models handle image and video input and output for productivity and education.