Conference Presentation, Keynote
Liran Zvibel, Cofounder, CEO of WEKA: Rise of the Token Economy The New Economics of AI
- 2025 is predicted to mark the dawn of artificial general intelligence and the fourth industrial revolution, driven by reasoning models, with leading models in this sector anticipated to reach valuations in the trillions of dollars.
- The current "race to the bottom" in AI token economics, which began approximately two years ago, will continue as solving advanced problems like ARC-AGI V2 riddles requires an increasing number of tokens, while OpenAI is expected to limit user usage to manage high inference costs until efficiency is achieved.
- Future infrastructure plans include implementing checks and balances to allow servers to handle both training and inference simultaneously using just-in-time training and inferencing, while addressing a significant predicted GPU memory bottleneck during inference where data is written out from HBM.
- Expected improvements involve extending effective device memory to run more cores concurrently and optimizing the pre-fill phase to separate it from the decode phase, potentially saving up to 95% of GPU cycles in multi-turn scenarios and ensuring profitability by minimizing theoretical cycles per output token.
- Low latency is projected to become critical for large-scale infrastructure, as rising latency could increase costs and force suboptimal decisions regarding quantization, whereas innovations in pre-fill and KV cache saving aim to guarantee low latency while scaling user numbers and practical AI outputs.