Performance Engineering on Hard Mode with Andrew Hunter

Performance Engineering Context: Scale and Leverage

• Hyperscalers (e.g., Google):
  • Optimization opportunities are abundant ("easy mode") because any reduction in CPU cycles yields massive financial returns due to immense scale.
  • The "data center tax" (10–20% of cycles spent on low-level infrastructure) represents a high-leverage target; optimizing common infrastructure (compilers, OS, memory allocation) benefits all diverse business logic.
  • Cost structure prioritizes the physical hardware (CPU count, power) over specific business logic, making infrastructure optimization the primary driver of value.
• Jane Street (Trading Systems):
  • Low Leverage for Micro-optimizations: Small improvements (e.g., 1% faster logging) do not significantly alter the cost dial due to lower total compute scale.
  • Latency over Utilization: Unlike web systems, trading systems prioritize low latency and burst handling over high CPU utilization; systems intentionally idle (spinning on polling) 95–99% of the time.
  • Thesis: The value of optimization shifts from broad infrastructure tax to specific, high-impact latency reductions in critical code paths.

Measurement Challenges and Tooling

• Sampling vs. Tracing:
  • Sampling Profilers (e.g., perf):
    • Works by interrupting execution (e.g., every 100 microseconds) to capture stack traces and hardware counters (cycles, cache misses, branch mispredictions).
    • Limitation: Fails to capture causality or timing of rare events; a system can appear fast statistically while suffering from microsecond-level misconfigurations (e.g., low-priority queues introducing 200µs delays).
    • Tail Latency: Useful for identifying hotspots but often misses "tail" latency causes which are frequently queueing issues (repeated median behavior during bursts) rather than algorithmic errors.
  • Tracing (e.g., Magic Trace):
    • Captures the complete sequence of events over a short window using Intel's hardware-integrated "Processor Trace" (ring buffer of CPU activity).
    • Advantage: Provides a deterministic, time-ordered record of execution, revealing exact call sequences, tight loops, and misconfigurations invisible to sampling.
    • Trade-off: Higher overhead (Intel PT estimated at 5–15% performance cost) and limited to short retrospective windows.
• Visualization Tools:
  • Flame Graphs:
    • Represent time spent as width of "flames" (stack traces).
    • Critical Flaw: Obscures "join points" where multiple diverse code paths converge on a single shared function (e.g., malloc), causing the aggregate cost of that function to be visually underweighted.
  • Pprof (DAG-based):
    • Visualizes functions as nodes and call paths as edges, highlighting where time converges.
    • Advantage: Makes shared bottlenecks (join points) obvious by aggregating all incoming call paths to a single node, revealing systemic issues like excessive memory allocation.

OCaml and Language Trade-offs

• Mechanical Sympathy Mismatch:
  • OCaml's boxed memory model (pointers to objects) lacks the raw memory locality of C/C++, creating a "representational gap" between the high-level language and the physical hardware.
  • Garbage collection introduces risks with uninitialized data, requiring strict discipline (e.g., nulling pointers) that adds overhead compared to manual memory management.
• Optimization Strategies within OCaml:
  • Zero-Allocation Dialects: A subset of OCaml code avoids the garbage collector and raw memory manipulation via custom data layouts to achieve performance comparable to C in hot paths.
  • DSLs and Interop:
    • Domain-Specific Languages (DSLs) generate type-safe interfaces backed by low-level, flat memory structures for parsing and data representation.
    • C interop is utilized for heavy lifting with low overhead (3–4 nanoseconds per call) compared to Java (300–400 nanoseconds).
  • Architectural Workarounds: Compensate for language limitations by reducing system hops, pre-processing data, and distributing logic across multiple processes to mitigate cache footprint issues.

Hardware and the OODA Loop

• Hardware vs. Software Roles:
  • FPGAs: Capable of sub-100ns latencies but are difficult to write (Verilog) and impractical for complex logic; reserved for hyper-specialized, simple functions.
  • CPUs: Necessary for complex decision-making; optimization focuses on making software fast enough to be competitive (e.g., 5–50µs tails) while outsourcing the absolute fastest tasks to hardware.
• Iteration Speed (OODA Loop):
  • Concept: Faster "Observe, Orient, Decide, Act" loops allow for more iterations, leading to better system quality and adaptability (e.g., rapid bourbon aging vs. traditional 10-year aging).
  • Application: Reducing analysis-to-feedback time from days to minutes (via testbeds) significantly accelerates the development of trading strategies and performance improvements.
  • Human Interaction: Low latency is critical for human-in-the-loop systems to maintain user engagement and the ability to iterate on creative ideas in real-time.

Expertise and Discipline

• Mechanical Sympathy: Defined as the unconscious, intuitive understanding of how code maps to hardware realities (CPU caches, branch prediction, memory layout).
  • Requires deep study of operating systems, compilers, and hardware architecture rather than abstract algorithmic theory.
  • Learned through experience and a compulsion to understand "boring" low-level details.
• Optimization Discipline:
  • Engineers must exercise restraint, prioritizing architectural changes over micro-optimizations (e.g., loop unrolling) that do not move the needle.
  • The hardest skill is ignoring local inefficiencies to focus on global system design and what not to do (avoiding expensive operations entirely).