Latest Interviews

Jane Street on GPUs, Trading, and Hiring: A Conversation with Dwarkesh

Jane Street, Dwarkesh Patel, Ron Minsky, Dan Pontecorvo

Jane Street executes a multi-tiered trading strategy that balances nanosecond-scale FPGA execution with long-horizon AI models while transitioning to a disaggregated global network featuring a $6 billion compute partnership with CoreWeave. The firm prioritizes model architecture diversity and custom hardware, including ARM and NVIDIA infrastructures, to navigate high-noise financial data without relying on single foundation models. This approach is supported by a hiring strategy focused on scarcity-driven talent acquisition and a human-in-the-loop philosophy that maintains critical judgment during complex market anomalies.

Production Engineering When Trading Billions of Dollars a Day

Mark Doss

Mark, a production engineer at Jane Street, outlines a high-stakes trading environment where even a 0.01% error rate can trigger insolvency, necessitating a monitoring strategy that rejects standard service level objectives in favor of code-level, event-based alerts. The firm employs a defense-in-depth approach using redundant, symptom-focused detection systems to catch catastrophic failures like fat-finger trades or stale market data before they cascade. By integrating deep domain knowledge into incident response and treating monitoring infrastructure as more critical than the trading systems themselves, Jane Street ensures that traders and engineers collaborate to resolve unique operational risks with extreme precision.

The Cost of Concurrency Coordination with Jon Gjengset

Jon Gjengset, John, Gabriel Kreiman

The presentation challenges the conventional view that mutexes are inherently slow, demonstrating instead that performance degradation in high-concurrency environments stems from CPU cache coherence overheads and MESI protocol costs rather than the lock mechanism itself. To address false sharing and serialization issues found in reader-writer locks, the speaker details the Left-Right data structure, a lock-free architecture that achieves linear scaling for read-heavy workloads by decoupling reader access from writer synchronization. Finally, the discussion emphasizes that optimal synchronization strategy depends on the specific read-to-write ratio and consistency requirements, urging developers to profile cache behavior and avoid blind optimization of lock primitives.

Bryan Cantrill: Andreessen’s Folly - The False Dichotomy of Software and Hardware

Bryan Cantrill, Andreessen, Alan Kay, Jonathan Gruber, Trevor Johnsen, Tred Wansell, Jermaine

Speaker critiques Marc Andreessen's "Software Eating the World" thesis as a false dichotomy while detailing Oxide Computer Company's hardware-software co-design strategy to reclaim stack control. By abandoning proprietary layers like UEFI and Baseboard Management Controllers in favor of custom operating systems and holistic integration with AMD silicon, the company addresses systemic reliability failures that plague commoditized reference designs. This approach has already resolved complex hardware-software interaction bugs and positions the firm to pursue open-source firmware and transparent accelerator partnerships to eliminate vendor lock-in in future data center infrastructure.

Arjun Guha: How Language Models Model Programming Languages & How Programmers Model Language Models

Arjun Guha

Arjun Guha presents a comprehensive analysis of large language models in programming, highlighting how traditional benchmarks are saturating while new methods like multi-PLE and language-agnostic transforms reveal significant performance gaps in low-resource languages such as OCaml. Through mechanistic interpretability techniques like activation steering, the talk demonstrates that internal model vectors can effectively correct type prediction errors and switch target languages without retraining, exposing shared representations across diverse syntaxes. These technical insights are contextualized by human studies showing that student success in prompting models hinges on providing specific semantic clues rather than syntactic fixes, while industry data reveals a surge in AI co-authorship alongside complex debates regarding actual productivity gains.

Making GPUs Actually Fast: A Deep Dive into Training Performance

Corwin, Savant Diaz, Sylvain De Wecker

Jane Street engineers optimize deep learning infrastructure by eliminating CPU-GPU synchronization bottlenecks and fusing PyTorch operations via `torch.compile` and Triton to maximize throughput. When automated compilation fails on complex Python logic, they deploy custom CUDA kernels that leverage shared memory and warp-level reductions to achieve nearly 1,000x speedups in specialized tensor operations. This hierarchical approach, ranging from standard PyTorch to hand-optimized C++, ensures efficient utilization of the H100's 132 Streaming Multiprocessors and strict memory bandwidth constraints.

Neil Mitchell: Pyrefly: Type Checking 1.8 Million Lines of Python Per Second

Neil Mitchell, Roman

Meta engineer Neil Mitchell introduced PyreFly, an open-source Python type checker reimplemented in Rust to address performance and scalability limitations for massive codebases like Instagram. The tool utilizes an aggressive memory eviction strategy and file-level concurrency to deliver rapid IDE feedback while supporting complex type features such as structural subtyping and flow narrowing. Released under the MIT license with over 100 contributors, PyreFly aims to replace legacy systems by prioritizing broad ecosystem adoption and seamless integration with build tools like Buck.

Hacking OCaml

Corwin de Zahr

In a 2021 CTF challenge titled "Secure OCaml Sandbox," the speaker demonstrates how to bypass type safety by exploiting exceptional polymorphism and runtime inconsistencies within the OCaml standard library. By leveraging functions like `callback.register` alongside low-level insights into integer tagging and closure memory layouts, the attacker constructs a custom `obj.magic` to redirect code execution toward blocked primitives such as `open_in`. This successful exploitation retrieves the hidden flag, illustrating how deep knowledge of the OCaml runtime can undermine soundness guarantees even when standard file and command execution are restricted.

Will Crichton: Rust for Everyone!

Will Crichton

Will Creighton's research at the Cognitive Engineering Lab applies human-centered design and formal cognitive theories to address fundamental learning and debugging barriers in Rust. By developing three core tools—Aquascope for visualizing ownership permissions, Argus for interactive trait solver trees, and Flow History for precise program slicing—the team achieved a 9-point score increase in learner assessments and a threefold speedup in error localization during user studies. Future efforts are now directed toward resolving async/await complexities and promoting extensible IDE frameworks like CodeMirror to further advance a scientific approach to programming language design.

The Saga of Multicore OCaml

After a nine-year development cycle, Jane Street successfully deployed Runtime 5, a multi-core garbage collector for OCaml that addresses initial performance regressions through a redesigned pacing model and a Mark Delay synchronization patch. The team resolved complex "load-bearing bugs" and floating garbage issues to achieve stable parallelism while maintaining the sequential performance required for high-frequency trading environments. Despite rejecting the ownership model of languages like Rust, Jane Street is now releasing its production-ready OxCamel branch to facilitate researcher experimentation on future parallelism tools and type-level race freedom guarantees.

Making OCaml Safe for Performance Engineering

Mark

Jane Street's research introduces a suite of OCaml extensions featuring unboxed types and stack allocation to eliminate memory waste and garbage collection overhead in performance-critical applications. These innovations expand into a static mode system that guarantees data race freedom for parallel execution by enforcing lifetime and portability constraints without explicit annotations. Currently deployed in production for memory management features and undergoing beta testing for concurrency safety, this work aims to integrate into mainline OCaml while earning a POPL award for its formal verification of race freedom.

Horace He: Building Machine Learning Systems for a Trillion Trillion Floating Point Operations

Horace He

Meta compiler engineer Horace He analyzes the dramatic consolidation of AI infrastructure, noting that modern model training now requires massive power resources and billions in capital to achieve state-of-the-art performance. He details how the industry has transitioned from simple imperative execution to complex compiler strategies like FlexAttention and `torch.compile`, which are essential for managing the critical balance between GPU compute and memory movement. Ultimately, He argues that the primary challenge in this field is shifting focus from pure optimization to designing robust programming models that allow developers to reliably express complex performance trade-offs in large-scale distributed systems.

System Jitter and Where to Find It: A Whack-a-Mole Experience

Tudor Brindus, Harry

A technical analysis of a simulated low-latency message passing system in OCaml demonstrates that moving from a virtualized environment to bare metal hardware reduces median round-trip latency from 637 nanoseconds to 223 nanoseconds. By sequentially isolating network interrupts with `isolcpus`, disabling timer ticks via `nohz_full`, locking CPU frequencies, and optimizing speculative execution, researchers achieved a final stable median latency of 163 nanoseconds with near-zero jitter. The study concludes that while aggressive overclocking offers marginal speed gains, it introduces unacceptable risks of memory corruption, establishing deterministic bare metal configurations as the necessary standard for reliable high-frequency trading infrastructure.

Charlie Marsh: uv: An Extremely Fast Python Package Manager

Charlie Marsh

Charlie Marsh of Astral introduces UV, a Rust-based Python package manager that functions as a unified drop-in replacement for tools like pip and Poetry by combining installation, dependency resolution, and environment management into a single high-performance workflow. Since its mid-February release, the tool has achieved 16 million monthly downloads by addressing Python's complex dependency graph challenges through advanced SAT solving, Algebraic Decision Diagrams, and a zero-copy architecture that treats virtual environments as ephemeral. UV now commands over 10% of all PyPI requests, leveraging custom version parsing and cache reflinking to enable near-instant package installation and declarative dependency locking.

The Uncertain Art of Accelerating ML Models with Sylvain Gugger

Sylvain Gugger, Ron Minsky, Jeremy Howard, Mark Mandelmann, Mark Mirchandani, Francesc Campoy, Gabriel Sanchez

Former fast.ai co-author Jeremy Howard discusses his transition from mathematics education to optimizing machine learning infrastructure at Jane Street, highlighting breakthroughs in learning rate schedules and image resizing that previously secured top benchmark placements. He details the development of the Hugging Face Accelerate library, a lightweight tool designed to abstract complex hardware parallelism and eliminate boilerplate code for training across diverse GPUs and TPUs. The discussion further explores the architectural constraints of financial data, the dominance of PyTorch's iterative execution model, and Jane Street's rigorous approach to reproducibility and custom model development for high-frequency trading.