Liran Zvibel, Cofounder, CEO of WEKA: Rise of the Token Economy The New Economics of AI

Macro Context & Economic Shifts

• 2025 is identified as the "dawn of artificial general intelligence" (AGI), marking the start of the fourth industrial revolution.
• The fourth revolution is defined by the transition from pattern-matching (third revolution) to reasoning models capable of solving complex, idea-based problems.
• Historical data shows a direct correlation between technology scaling, adoption, and GDP growth across the steam engine, electricity, microprocessor, and AI eras.
• The "tokenomics" of AI—specifically the economics of tokens—are becoming a critical differentiator for the industry's direction and profitability.

Strategic Spend: Innovation vs. Inference

• AI Innovation: Major entities (OpenAI, Anthropic, "Rock," Meta) are investing "ridiculously high" capital to train models and poach researchers, viewing leadership in the next generation of models as a potential trillion-dollar opportunity.
• AI Inference: Even leading innovators like OpenAI are not prioritizing "inference at all costs"; Sam Altman recently noted that costs must be reduced before scaling image generation features further.
• The consensus among industry leaders is that while training requires heavy investment, inference must be solved economically to be viable at scale.

Technical Hurdles & Efficiency Metrics

• ARC-AGI Benchmark: Created by Google's François Chollet, this test distinguishes general intelligence (solving tasks not seen in training) from specific skills.
  • First-generation (yellow) riddles were mostly solved only after GPT-4.
  • Second-generation (blue) riddles remain unsolved by current models like Grok and O3.
  • Humans solve these riddles instantly with zero token cost, highlighting the exponential token cost growth for AI as it approaches human-level reasoning.
• Profitability Condition: SemicAnalysis data confirms that increasing model performance correlates with potential profit, but only if efficiency in the token economy is maintained.
• Recent breakthroughs (e.g., "Deep Sea" model) demonstrate that complex reasoning tasks can be solved with significantly lower costs than previous iterations.

Infrastructure Solutions & Decisions

• Server Elasticity: New infrastructure allows the same GPU servers to alternate between training and inference; servers can checkpoint training and quickly reload for inference during user peak times to maximize utilization.
• Memory Bottlenecks: Extending effective HBM (High Bandwidth Memory) is critical to increase concurrent cores and user capacity for inference.
• Latency as a Cost Driver: Latency is identified as a "sign killer" for AI success; higher latency forces inefficient quantization choices and directly increases infrastructure costs while reducing accuracy control.
• Pre-fill Disaggregation: A key architectural shift involves separating the pre-fill phase (attention calculation) from the decode phase (output generation).
  • In multi-turn conversations, pre-filling can result in up to 95% waste by recalculating the same attention 400+ times.
  • Implementing KVCache storage and reloading mechanisms allows systems to avoid redundant pre-fill calculations, theoretically minimizing cycles per output token.

Forward-Looking Statements & Vendor Positioning

• The speaker asserts that "just-in-time" training and inferencing are necessary to make AI sustainable and successful.
• The company "Weka" positions itself as the only storage infrastructure created specifically for AI, focusing on:
  • Reducing AI costs dramatically.
  • Guaranteeing low latency.
  • Enabling pre-fill and KVCache saving mechanisms to scale practical AI usage.
• The ultimate goal is to drive the industry toward profitability by ensuring infrastructure costs align with the actual volume of generated tokens.