Dwarkesh Goes Inside Jane Street's Latest AI Data Center

Operational Scope

• The facility hosts a training cluster utilizing GP300 and VL72 GPU units, currently executing diverse workloads:
  • Training Large Language Models (LLMs).
  • Running custom architectures optimized for trading problems and datasets.
• The site contains 4,032 GPUs distributed across 56 racks.

Infrastructure Retrofit & Cooling

• The Texas facility was retrofitted from a traditional air-cooled design (10–40 kW per rack) to support high-density liquid cooling (up to 140 kW per GP300 cabinet).
• Approximately 85–90% of the current load is liquid-cooled via cold plates on GPU sleds; the remaining 10–15% utilizes legacy air-cooling equipment.
• Cooling architecture details:
  • Fluid enters at approximately 18°C from rooftop chillers.
  • A CDU (Coolant Distribution Unit) manages flow balance using ultrasonic flow meters to ensure equal distribution (liters per minute) across cabinets, preventing starvations or overflows.
  • The secondary "technical water loop" uses a mixture of distilled/deionized water and 25% propylene glycol to inhibit bacterial or algae growth that could clog cold plates.
  • The loop includes filtration down to 25 microns to protect heat exchange efficiency.
• Leak detection systems include:
  • Ropes placed under cabinets and in the raised floor to detect drips.
  • Automated alerts from the switch management system.
  • Valves capable of isolating specific sections upon leak detection.
• Redundancy measures include buffer tanks acting as a "thermal battery" to maintain GPU cooling during chiller interruptions or power restarts.

Power Distribution & Load Management

• Power distribution utilizes overhead busways and conduit to support high-density loads while respecting utility capacity limits; the facility remains "grid-connected" rather than behind-the-meter.
• Engineering strategy involves "fungibility," allowing power to be redistributed across rows to accommodate flexible growth (e.g., shifting capacity between CPU and GPU zones).
• Critical power risks involve tripping breakers due to:
  • Exceeding amperage limits on single busways.
  • Oversubscription scenarios where peak loads exceed the anticipated 10% threshold.
• Software-driven safeguards include:
  • Proprietary monitoring tools providing a unified "pane of glass" for real-time topology awareness.
  • Automated logic to shut down specific nodes or workloads if power draw approaches critical thresholds, preventing total system interruption.
  • Adoption of NVIDIA's load management systems (LPS) to flatten load profiles using bulk capacitance in power shelves.
• Hardware costs are significant, but the "opportunity cost" of compute downtime often dominates financial considerations due to the high value and scarcity of training resources.

Networking & Latency Optimization

• The deployment includes approximately 8,000 kilometers of fiber optic cabling.
• Critical internal connections utilize copper cabling over fiber to maximize signal velocity, as electrons in copper move faster than light in fiber.
• Networking rig complexity is heightened by density; overhead piping is preferred over raised floors to accelerate deployment speed.

Historical Context & Evolution

• Jane Street's computing infrastructure has evolved from a "Hive" cluster (six stacked Dell boxes physically located in the trading room) to a dedicated, secure data center.
• Early trading systems prioritized immediate human access for shutdowns, leading to incidents where office cleaning staff inadvertently unplugged critical hardware.
• Latency requirements have shifted dramatically:
  • Early systems operated on second-to-millisecond scales.
  • Current top-tier systems require packet turnaround times under 100 nanoseconds.
• The ratio of supporting infrastructure (transformers, chillers) to compute space has increased significantly compared to 20 years ago, reflecting the higher density and power demands of modern hardware.

Strategic Insights

• The facility was designed with "optionality" in mind to accommodate uncertain future compute shapes, requiring extensive planning to balance multiple potential futures.
• High-density computing allows for smaller data hall footprints relative to power allocation, creating unused space that can be repurposed (e.g., for a podcast studio).
• Liquid cooling introduces new failure modes (leaks, biological growth) that did not exist in previous air-cooled environments, though engineering mitigations are currently in place.