Fireside Chat, Panel
Why the most demanding companies entrust their AI to OCI & NVIDA
- Blackwell generation GPUs are projected to deliver training performance four times higher than Hopper, while inference performance is expected to be 30 times higher with 25 times less energy consumption.
- The GH200NVL72 rack-scale solution, consisting of 72 interconnected chips, is scheduled for delivery to OCI next year.
- Rapid technology evolution where new GPUs may offer up to 30 times the performance within a single year creates potential infrastructure investment discomfort for customers relying on on-premises hardware.
- OCI offers customers the flexibility to modernize infrastructure over one, two, or three-year cycles to maintain pace with technology without future baiting.
- Oracle commits to data center security, privacy, and sovereignty in Europe through operations in Frankfurt and Madrid managed by a local European team and encryption assurance via a partnership with Thales.
- OCI has recorded quarterly IaaS growth exceeding 50% in the US and plans to replicate this expansion in Europe, supported by substantial GPU capacity availability by this summer.
- Network architecture utilizing layer two virtualization and Rocky RDMA over Ethernet version 2 is designed to support clusters scaling up to 32,000 nodes.
- Microsoft may be executing Bing search inference on OCI driven by the price-performance ratio of the technology stack.
- The L40S GPU, expected to arrive by June, offers compute performance comparable to the A100 at a price point between the A100 and H100, targeting metaverse, graphics, and inference workloads.
- The L40S features core and transformer engines optimized for all LLMs and generative AI, positioned as an effective solution for production-grade LLM inference.
- NVIDIA Inference Microservices (NIMS) will bundle CUDA libraries and Triton into a single pre-optimized container to accelerate production deployment.
- NVIDIA frameworks are anticipated to increasingly address specific challenges in language models, computer vision, and healthcare as customers scale to production.
- Oracle version 23c includes vector database capabilities within the standard database to deliver significant performance and scalability benefits beyond demo scales.
- A partnership aims to develop a full-stack AI solution encompassing the entire pipeline from data acquisition and preparation to prompt generation with enterprise data access.
- Acceleration of database technologies in France is expected by NVIDIA in the near future.