Lecture, Interview
Stanford CS153 Frontier Systems | Scale, AGI, and the Future of Everything
- The fundamentals of startup creation have shifted significantly since 2014, necessitating updates to educational curricula, while affordable token expenditure now allows startups to replicate the output previously requiring a hundred-person engineering team.
- A multi-trillion dollar market is predicted to emerge soon from problems currently solvable only through automated coding, with the most promising opportunities being those that are currently "totally not obvious" to the broader market.
- Scaling laws for AI models, research teams, and company economics are expected to continue delivering returns far beyond consensus, driving ambition, speed, and task volume to levels previously unattainable for new ventures.
- Y Combinator's network effects and unique value are described as emergent properties that only exist at scale, having been absent at one-tenth or one-hundredth of the organization's current size.
- The current AI research pipeline involving pre-training, mid-training, post-training, and RL/SF loops is viewed as a temporary state expected to undergo a "major rewrite" at an unknown future time.
- By September of the current year, OpenAI is projected to utilize 500,000 A100 equivalent GPUs, with an expectation that AI will progress exponentially for at least another three and a half years.
- AI research capabilities are predicted to advance to the point where researchers can figure out complete new architectures in an end-to-end manner by March 2028.
- By September of the current year, OpenAI will utilize 500,000 A100 equivalent GPUs for AI research, while Codex is expected to reach a real inflection point around early this year or with version 5.5.
- Intelligence is anticipated to become a ubiquitous utility where organizations plug into an "OpenAI token subscription," similar to how consumers view airtime and gigabytes, as users care more about access and cost than underlying hardware.
- Frontier labs are expected to evolve into "inference companies to a significant degree," with compute costs and demand fluctuating based on supply and market conditions.
- Demand for AI is predicted to be "uncapped" if models become sufficiently smart and cheap, potentially driving users to run 10 to 100 personal agents simultaneously.
- Compute shortages are expected to persist potentially "forever" as long as progress continues, with the spread between long-term reservations and spot prices for H100 and Blackwell chips predicted to reach as high as 5x.
- Despite an anticipated "tsunami of hardware," the demand tsunami is expected to be even larger, making the current shortage feel comparable to the "COVID for the compute era."
- OpenAI faces the risk that scaling deep learning will result in failure if incorrect, and systems are predicted to break at an accelerating and unpredictable rate as they scale.
- Human cognition is not naturally evolved to process exponential growth in scaling laws, revenue, and organizational complexity, requiring significant time to reason through these dynamics.
- The education system is predicted to fail to adapt to a post-ChatGPT world without redesign, which could lead to significant atrophy of critical thinking skills.
- There is a fear that labor's leverage will shift to capital, necessitating ownership stakes in capitalism, such as a citizen's wealth fund, to manage the resulting economic shifts.
- There is an 80% probability that technology will become widely democratized in the future, driven by the necessity to avoid the concentration of wealth and power.
- AI is predicted to surpass human intelligence in a much broader distribution of tasks than currently realized, including the ability to discover or disprove conjectures previously thought impossible.
- Specific product timelines indicate that ChatGPT was expected to become a "killer app" for general models within five days of observing traffic, while coding is predicted to become the "killer enterprise app."