Conference Presentation, Keynote
From Metal to Model: Why Operations Win the AI Infrastructure Race | Mirantis | RAISE Summit 2026
- Core Thesis: Success in future GPU clouds relies on the combination of platform, process, and full-stack ownership, as hardware commoditization shifts the durable competitive advantage to operations.
- All future IT infrastructure is defined as AI infrastructure, requiring systems that support integrated intelligence workflows.
- Workload Shift: Inference has become the dominant GPU workload, surpassing training in scale and cost.
- Inference currently accounts for ~66% of all AI compute and is projected to reach 75% by 2030.
- OpenAI's 2024 inference spending reached $2.3 billion, approximately 15 times the cost of training GPT-4.
- Efficiency gains drive increased demand (Yevon's Paradox); for example, after DeepSeek trained on 2,048 GPUs, Stargate announced nearly $500 billion in additional capacity spend.
- Infrastructure Divergence: Training and inference require fundamentally different architectural approaches.
- Training: Prioritizes sustained high throughput, tight interconnects (East-West RDMA), fast failure recovery, and checkpoint reliability.
- Inference: Prioritizes ultra-low latency, multi-tenancy, elastic workload scaling, and cost-per-token optimization.
- Real-time: Focuses on user experience and latency (e.g., ChatGPT response speed).
- Throughput: Optimizes for batch processing volume (e.g., large corpus analysis).
- Cost-optimized: Handles background tasks like vector store updates.
- Operational Failure Trends: Cluster reliability degrades significantly as scale increases.
- Meta observed failures roughly every 3 hours in training jobs (networking and optics being primary culprits).
- In large clusters (25k-100k GPUs), optic and network failures occur every 30 minutes.
- Approximately 1 in 400 failures (0.25%) requires human intervention for hardware replacement (optics, GPUs, nodes).
- Pillar 1: Observability: Critical for predicting and preventing failures in massive scale environments.
- Telemetry generation reaches hundreds of gigabytes per day in 25k clusters and terabytes in 100k clusters.
- Early detection is possible via low-level signals; e.g., a slight temperature uptick in optics can predict failure within 24 to 48 hours.
- Industry standard requires exposing low-level telemetry to operators for trend detection and automated routing.
- Pillar 2: Process & Procedures: Operational workflows directly impact financial performance and uptime.
- Improving MTTR (Mean Time To Recovery) and MTBF (Mean Time Between Failures) reduces GPU idle time.
- A specific cloud provider achieved a >40% increase in uptime solely by implementing rigorous change management, access control, and runbooks.
- Processes must be established prior to cloud launch, not retroactively.
- Pillar 3: Full-Stack Ownership: Avoiding "black box" vendors is essential for reliability and cost control.
- Lack of visibility into any layer (silicon, networking, storage, OS) prevents failure prediction and management.
- Open ecosystems and orchestration allow for dynamic adjustment of topology, hardware selection, and model architectures.
- Flexibility is required to handle the rapid evolution of model architectures and workload types faster than infrastructure build-out.
- Proposed Solution: Mirantis is launching Cordoned AI, a metal-to-model platform designed to support both CPU and GPU resources.
- The platform utilizes an open, standard-driven architecture with open APIs for the industry.
- It aims to provide the necessary visibility and automation to manage the full stack from data center power to model inference.