Stanford MS&E435 Economics of the AI Supercycle | Spring 2026 | Enterprise Internal Knowledge

Career Trajectory & Background

• Yash Patil, founder/CEO of Applied Compute, is a Stanford Class of 2025 graduate who bypassed traditional internships to work directly for OpenAI's post-training team starting in early 2023.
• Patil's connection with Sam Altman originated from a blind email during freshman summer, receiving initial seed funding for a project that was later shut down before leading to his OpenAI residency.
• He co-founded Applied Compute approximately one year ago after identifying a market gap where general "frontier" models lacked enterprise-specific data integration and business context.

Model Architecture & Training Evolution

• AlexNet (2012): Marked the pivot from handcrafted features to deep learning, proving that scaling compute and data (e.g., ImageNet) drastically improved predictive accuracy.
• Transformers (2017-2018): Google Brain's architecture introduced self-attention mechanisms, enabling better handling of long sequences and massive scaling on GPUs compared to RNNs/LSTMs.
• Scaling Laws:
  • Kaplan Laws: Demonstrated that model size is a primary driver of general intelligence (GPT-3).
  • Chinchilla Laws: Proved that optimal scaling requires increasing both parameter count and training data volume simultaneously.
• Reasoning Models (2024): OpenAI's O1 introduced "test-time compute," utilizing chain-of-thought reasoning as an emergent behavior rather than explicit training, allowing models to self-correct and solve complex problems.

Data Dynamics & Bottlenecks

• Pre-training Data Scarcity: The industry is approaching the limit of high-quality internet-scale text; future pre-training will rely on synthetic data generation, ancient book digitization, and architectural efficiency.
• Bottleneck Shift: The primary constraint has moved from compute to continual learning—the ability to learn from extremely sparse, real-world rewards (e.g., burning a hand once) rather than massive pre-training datasets.
• Data Sources: Frontier labs now face an "economy of data" where proprietary enterprise data becomes the primary differentiator, as public internet data is exhausted.
• Synthetic Data: As models improve, RL environments increasingly rely on synthetic verifiable rewards (e.g., unit tests) to reduce reliance on human-labeled data.

Specialization vs. Generalization (Applied Compute)

• The Gap: General models act as "geniuses" with no knowledge of specific business rules; enterprises require specialized models trained on proprietary data to close this gap.
• Case Study (DoorDash): Applied Compute built a specialized Vision Language Model (VLM) for menu extraction that outperformed general models by optimizing directly against specific error rates and style guides rather than relying on prompting.
• Case Study (Cognition/Windsurf): Developed a specialized model for bug detection in code, offering sub-2-second latency and lower costs by focusing on a narrow, high-frequency task.
• Case Study (Ramp): Trained a model for fast spreadsheet search, integrating RL to optimize specific user experience metrics.
• Compute Economics:
  • Pre-training requires ~100x more compute than post-training (e.g., DeepSeek V3 required ~2.4M H100 hours for pre-training vs. ~150k for RL).
  • The trend indicates RL training compute is rising relative to pre-training as labs adopt multi-datacenter RL runs to maximize performance via scaling laws.

Emerging Techniques & Future Trends

• Continual Learning:
  • Cursor's "Composer": Implemented online learning where implicit rewards (user acceptance vs. reversion of code suggestions) drive model updates within days or weeks.
  • Context Space: Using agents offline to analyze historical traces and documents to extract learnings for downstream performance improvements.
• Evaluation (Evals):
  • Evals serve as the "roadmap" for model development; teams optimize training pipelines specifically to maximize scores on benchmarks like SWE-bench.
  • Enterprises maintain internal evals to define "good vs. bad" outputs unique to their specific operational standards (e.g., finance vs. retail).
• Architecture Debate: Patil remains skeptical of non-transformer models (e.g., Mamba) challenging the Transformer's dominance in the near term, citing the high capital investment in current chip infrastructure and the proven success of scaling Transformers.
• Agentic Coding: Code is the primary frontier for reasoning models because it offers deterministic verifiable rewards (compilation, unit tests) and serves as a "universal language" for interacting with the real world.

Investment Stances & Market Predictions

• Long Bets:
  • Compute Hardware: Bullish on Nvidia and chip makers due to the massive disparity between compute demand and supply; expects continued co-development between model training and chip architecture.
  • Counter-argument Risk: Large labs may eventually in-house chip design to avoid the ~75% margin on current GPUs, though this remains a long-term, difficult transition.
• Short Bets / Skepticism:
  • Data Market: Skeptical of the long-term viability of pure data-selling businesses, as improving models make it harder to generate high-value synthetic tasks and increase the cost of creating new evaluation metrics.
  • Prediction: The data market will pivot toward robotics, egocentric data (GoPro style), and specialized RL environments rather than static text datasets.
• Future Focus: If not building Applied Compute, Patil would pursue hardware innovation to optimize the co-development of energy-efficient chips and model training.

Personal Preferences

• Favorite AI Product: Image-to-Image (Image Two), specifically for its ability to visualize concepts and turn rough sketches into structured designs.
• Definition of AGI: Views AGI as a future state requiring "loud feedback" loops (one-time learning from sparse rewards) rather than the current paradigm of massive data ingestion.