WEKA, Cohere, Nebius: Rewriting the Rules Agentic AI and the Race to Scale

Panel Overview and Context

• The session features Lauren Vaccarello (CMO, Weka), Cécile Robert-Michon (Head of Technical Teams, Cohere), and Dan Galpin (CTO, Nabeus) discussing AI infrastructure, scale, and economics.
• Key themes identified include the necessity of massive scale, the need to "rewrite rules" for an AI-first world, and the economic viability of commercial AI.

Scaling Challenges and Speed

• Cécile Robert-Michon described the current AI development pace as akin to "building rail tracks as a high-speed train approaches," necessitating constant iteration and rethinking of infrastructure.
• Cohere faces unique challenges in testing and integrating open-source projects hours after release and benchmarking GPU chips before their commercial availability.
• The shift to agentic AI has created a spike in inference demand, requiring optimizations that make individual tokens "smarter" rather than merely serving more tokens.

Flexibility vs. Performance Trade-offs

• Dan Galpin refuted the industry belief that flexibility requires compromising performance, stating the answer is "no."
• Nabeus avoids virtualizing GPUs and infinite bandwidth interconnects to prevent performance hits, achieving 99.7% (and sometimes 104%) of NVIDIA's benchmark standards on their platform.
• The company maintains a "software-first" architecture to support a wide range of users, from individual researchers with single-GPU budgets to enterprises with thousands of interconnecting GPUs.
• Galpin noted that roadmaps are intentionally short-term (approximately two weeks), with longer-term plans described as "fog," as rigid planning leads to failure in the rapidly evolving AI landscape.

Security, Privacy, and Enterprise Adoption

• Cohere prioritizes security and data privacy as core infrastructure decisions, specifically targeting regulated industries such as finance, healthcare, manufacturing, public sector, and energy.
• The company's "North" platform is designed to be cloud-agnostic and deployable in private, air-gapped environments to prevent data movement while maintaining compliance.
• Contrary to the perception that bleeding-edge innovation requires security trade-offs, Robert-Michon stated Cohere specializes in integrating high-level security into their foundation models and end-to-end products.
• Cohere's infrastructure strategy involves training on a fully cloud-native stack to build partnerships with major providers while maintaining technology neutrality.

Economics and Commercial Viability

• Dan Galpin argues that during the initial adoption phase, the priority should be capability and speed-to-market rather than immediate cost optimization.
• Galpin posits that "intellect per token" is a better metric than balancing costs in early iterations, with cost optimizations (such as smaller, specialized models) occurring later in the ecosystem lifecycle.
• Robert-Michon counters that software efficiency can unlock performance gains that raw compute cannot, stating that "maximizing efficiency and performance go hand in hand."
• Cohere recently collaborated with Weka to create a converged storage and compute layer, reusing existing GPU servers and NVMe disks to achieve a 20x improvement in checkpoint performance over traditional object storage without new resources.

Forward-Looking Advice and Strategic Shifts

• Speakers advise enterprises to "unlearn" legacy software assumptions and approach AI development with a "clean sheet" mindset.
• Robert-Michon highlighted that enterprise adoption is currently lagging behind consumer adoption, creating a significant gap for providers to solve regarding legacy stack integration.
• Galpin warned that many organizations claiming readiness for current AI scale from past two years are "lying," as the performance demands for storage, networking, and GPUs have reached previously unimaginable levels.
• The consensus is that agility is essential because the infrastructure landscape may shift significantly within weeks, making long-term rigid planning obsolete.