Latest Interviews

Production Engineering When Trading Billions of Dollars a Day

Mark Doss

Mark, a production engineer at Jane Street, outlines a high-stakes trading environment where even a 0.01% error rate can trigger insolvency, necessitating a monitoring strategy that rejects standard service level objectives in favor of code-level, event-based alerts. The firm employs a defense-in-depth approach using redundant, symptom-focused detection systems to catch catastrophic failures like fat-finger trades or stale market data before they cascade. By integrating deep domain knowledge into incident response and treating monitoring infrastructure as more critical than the trading systems themselves, Jane Street ensures that traders and engineers collaborate to resolve unique operational risks with extreme precision.

The Cost of Concurrency Coordination with Jon Gjengset

Jon Gjengset, John, Gabriel Kreiman

The presentation challenges the conventional view that mutexes are inherently slow, demonstrating instead that performance degradation in high-concurrency environments stems from CPU cache coherence overheads and MESI protocol costs rather than the lock mechanism itself. To address false sharing and serialization issues found in reader-writer locks, the speaker details the Left-Right data structure, a lock-free architecture that achieves linear scaling for read-heavy workloads by decoupling reader access from writer synchronization. Finally, the discussion emphasizes that optimal synchronization strategy depends on the specific read-to-write ratio and consistency requirements, urging developers to profile cache behavior and avoid blind optimization of lock primitives.

Matt Godbolt: Advanced Skylake Deep Dive

Matt Godbolt, Jesper

Matt Godbolt, a prominent C++ developer transitioning to HRT, presents a detailed reverse-engineered analysis of the Skylake-era CPU microarchitecture based on community findings rather than official documentation. The talk dissects critical pipeline stages including the front-end's instruction decoding, the micro-op cache limitations, and the complex register renaming mechanics that define the processor's performance characteristics. Key revelations include specific hardware flaws like the Loop Stream Detector bug, port allocation strategies, and the diminishing returns of increasing architectural register counts compared to the hundreds of physical registers already available.

Arjun Guha: How Language Models Model Programming Languages & How Programmers Model Language Models

Arjun Guha

Arjun Guha presents a comprehensive analysis of large language models in programming, highlighting how traditional benchmarks are saturating while new methods like multi-PLE and language-agnostic transforms reveal significant performance gaps in low-resource languages such as OCaml. Through mechanistic interpretability techniques like activation steering, the talk demonstrates that internal model vectors can effectively correct type prediction errors and switch target languages without retraining, exposing shared representations across diverse syntaxes. These technical insights are contextualized by human studies showing that student success in prompting models hinges on providing specific semantic clues rather than syntactic fixes, while industry data reveals a surge in AI co-authorship alongside complex debates regarding actual productivity gains.

Neil Mitchell: Pyrefly: Type Checking 1.8 Million Lines of Python Per Second

Neil Mitchell, Roman

Meta engineer Neil Mitchell introduced PyreFly, an open-source Python type checker reimplemented in Rust to address performance and scalability limitations for massive codebases like Instagram. The tool utilizes an aggressive memory eviction strategy and file-level concurrency to deliver rapid IDE feedback while supporting complex type features such as structural subtyping and flow narrowing. Released under the MIT license with over 100 contributors, PyreFly aims to replace legacy systems by prioritizing broad ecosystem adoption and seamless integration with build tools like Buck.

Will Crichton: Rust for Everyone!

Will Crichton

Will Creighton's research at the Cognitive Engineering Lab applies human-centered design and formal cognitive theories to address fundamental learning and debugging barriers in Rust. By developing three core tools—Aquascope for visualizing ownership permissions, Argus for interactive trait solver trees, and Flow History for precise program slicing—the team achieved a 9-point score increase in learner assessments and a threefold speedup in error localization during user studies. Future efforts are now directed toward resolving async/await complexities and promoting extensible IDE frameworks like CodeMirror to further advance a scientific approach to programming language design.

Making OCaml Safe for Performance Engineering

Mark

Jane Street's research introduces a suite of OCaml extensions featuring unboxed types and stack allocation to eliminate memory waste and garbage collection overhead in performance-critical applications. These innovations expand into a static mode system that guarantees data race freedom for parallel execution by enforcing lifetime and portability constraints without explicit annotations. Currently deployed in production for memory management features and undergoing beta testing for concurrency safety, this work aims to integrate into mainline OCaml while earning a POPL award for its formal verification of race freedom.

How OCaml Represents Values in Memory

Nalen

This presentation details OCaml's critical distinction between immediate and boxed types, explaining how tagged integers and heap-allocated blocks with specific headers enable efficient memory layout and garbage collection. It outlines the precise bit-level encoding strategies for primitives, variants, and special types like floats and strings, while noting the runtime's ability to dynamically inspect values and its constraints on non-immediate constructor counts. The discussion concludes by affirming the current system's performance benefits and previewing future optimization efforts toward integrating unboxed types.

Horace He: Building Machine Learning Systems for a Trillion Trillion Floating Point Operations

Horace He

Meta compiler engineer Horace He analyzes the dramatic consolidation of AI infrastructure, noting that modern model training now requires massive power resources and billions in capital to achieve state-of-the-art performance. He details how the industry has transitioned from simple imperative execution to complex compiler strategies like FlexAttention and `torch.compile`, which are essential for managing the critical balance between GPU compute and memory movement. Ultimately, He argues that the primary challenge in this field is shifting focus from pure optimization to designing robust programming models that allow developers to reliably express complex performance trade-offs in large-scale distributed systems.

How to Use OCaml's Coercion Operator

camels

This session explores standard OCaml's subtyping and coercion mechanisms, emphasizing how the `:>` operator enables compile-time type checks without runtime overhead or data reallocation. The discussion details value inclusion principles for polymorphic variants, recursive subtyping relations, and the specific constraints imposed by private type abbreviations to enforce invariants. While confirming that list coercions remain free at runtime, the presentation notes upcoming analysis of variant annotations and object types in future sessions.

Programming with OCaml's Local Mode | OCaml Unboxed

Jane Street Doe Camel

A Jane Street team member demonstrates practical limitations of the OCaml "Locals" region-based memory management system while optimizing the `best_of_prime_widgets` function to avoid heap allocation. To resolve type mismatches where local lists must escape their regions, the implementation employs a custom zero-overhead `global` wrapper type alongside a recursive map function annotated with `exclave` constraints. Although the solution successfully maintains local allocation for list structures while treating elements as global, the speaker identifies these manual workarounds as sharp corners requiring future improvements through mode polymorphism.

Pitfalls with Tail Calls and Locals in OCaml | OCaml Unboxed

Goldfirere

Jane Street researchers developed a "local mode" for their OCaml compiler to optimize memory allocation on the stack, but discovered that standard tail call optimization causes "local value escapes" errors when recursive closures capture variables from a region that ends immediately before the call. To address this, the team introduced a "regional" sub-mode that permits specific values to escape one region boundary, allowing tail-recursive functions to maintain $O(1)$ stack space without explicit `non-tail` annotations, though passing these values through intermediate functions can strip this status and force $O(n)$ allocations. The developers acknowledge that current workarounds like explicit annotations or variable indirection are cumbersome for practical use, prompting a push for better compiler heuristics to automate safe tail calls in future upstream releases.

Inferring Locality in OCaml | OCaml Unboxed

The OCaml compiler utilizes an internal allocation discipline that infers local versus global modes for variables and parameters to enable stack-based memory usage and reduce garbage collection overhead. This system prioritizes local inference for arguments while defaulting return values to global to ensure maximum compatibility, with integers serving as a special case that can safely cross mode boundaries under specific annotations. Although this framework enhances performance, current limitations such as standard library functions lacking locality awareness and the need for explicit type signatures to trigger safety checks continue to shape its practical application.

Annotating OCaml Variables and Returns with local_ | OCaml Unboxed

Jane Street is developing an experimental OCaml feature set that enforces strict region constraints through `local` annotations to optimize memory usage and eliminate escaping values. The system introduces the `exclave` keyword to terminate function regions early, enabling specific allocations in the caller's scope while preventing mutable references from capturing local data. Current implementation faces known pitfalls regarding return position rules and compiler error precision, with further refinements planned to address these sharp edges.

Real Numbers – Episode 16, Finale

Sawyer

The final episode of Season 1 of *Real Numbers* derives five distinct mathematical solutions to the geometric distribution problem of calculating the expected number of half-court shots Danielle needs to make. The episode further analyzes a bonus problem involving two consecutive successes, demonstrating via a recursive state method that the expected attempts equal 30 rather than the intuitive estimate of 25. Concluding the season, the host solicits listener feedback on problem topics, format changes, and potential mathematical depth for the upcoming second season.