Jane Street on GPUs, Trading, and Hiring: A Conversation with Dwarkesh

Jane Street's trading operations span a multi-tiered latency spectrum:

• Ultra-low latency strategies require packet turnaround under 100 nanoseconds, utilizing direct FPGA attachments where CPU usage is impossible and decisions must be simple.
• Medium-frequency strategies operate on timescales ranging from microseconds to milliseconds.
• Long-horizon strategies accommodate turnaround times of hours or days for complex, data-intensive decisions.
• The firm employs an ensemble approach, balancing decision intelligence against execution speed across these distinct time horizons.

AI and machine learning applications at Jane Street differ fundamentally from traditional foundation models:

• The firm prioritizes model architecture diversity and rapid experimentation over training single, general-purpose "foundation" models.
• Prediction targets focus on "fair value" estimation to enable composable integration into various trading processes.
• Financial data is characterized by high volume, high noise, and lower bits-per-flop informative density compared to natural language datasets.
• The firm maintains strict specialization in data consumption pipelines to handle varying data rates and ingestion methods.

Physical infrastructure and compute procurement strategies include:

• A $6 billion compute deal was signed with CoreWeave to support scaling laws and accelerate model iteration times.
• Inference hardware placement is dictated by latency requirements:
  • Sub-nanosecond systems require physical proximity to exchanges, often involving precise fiber length measurements.
  • Larger, slower models allow for greater physical flexibility, though co-location introduces power, cooling, and space constraints.
• The firm is transitioning from a single data center and x86-only architecture to a disaggregated global network supporting ARM and NVIDIA architectures.
• Internal teams are building a custom large-scale object store to manage distributed data across multiple facilities.

Data center engineering trends and constraints:

• The industry is shifting toward modular data centers and pre-built infrastructure to mitigate long lead times for critical components like generators and liquid cooling systems.
• Power density is increasing, with racks now approaching one megawatt, necessitating advanced cooling and power delivery solutions (e.g., 800V DC).
• Procurement strategies involve stocking fungible components and staging long-lead items (e.g., turbines) to ensure six-month acceleration on GPU deployments.
• The firm partners with hardware vendors to ensure component compatibility with future high-density standards.

Operational philosophy regarding AI and human-in-the-loop systems:

• The firm rejects the notion that AGI will immediately automate trading; trading is described as an "NP-complete" problem involving diverse, non-electronic variables and market phase transitions.
• Human judgment remains critical during market anomalies ("phase transitions") where models may underperform.
• Non-electronic trading persists, particularly in bond markets and chat-based execution, requiring human intermediation and judgment.
• Hiring is driven by the scarcity of high-caliber talent rather than compute availability; the firm aims to grow from tens of thousands to hundreds of thousands of GPUs to support this expansion.

Strategic investment priorities and hiring focus:

• Compute is viewed as a primary constraint; the firm has high-value unused capacity for retraining models and bulk inference tasks.
• The firm is hiring across physical engineering (mechanical, electrical, structural), machine learning, and software engineering, with a specific focus on fleet-wide optimization.
• New strategic investments include:
  • Custom ASIC development.
  • A formal methods team using mathematical proofs to verify software.
  • Enhanced front-end tooling to improve human productivity and agency.
• The firm utilizes puzzles and challenges (available at janestreet.com/dworkesh) as a cultural signal to attract talent, including unsolved challenges related to LLM backdoors.