Conference Presentation, Fireside Chat, Interview
The Saga of Multicore OCaml
- Jane Street undertook a nine-year project to onboard a multi-core garbage collector (Runtime 5) for OCaml, transitioning from a single-core runtime (Runtime 4) to a shared-memory parallel system.
- The primary goal was to introduce parallelism without degrading sequential performance, maintaining existing pause times, and preserving compatibility with the Foreign Function Interface (FFI).
- A 2020 paper titled "Retrofitting Parallelism onto OCaml" established the initial design, proposing a single unified runtime optimized for both single-core and multi-core workloads.
- Despite the 2022 upstream release of the new GC, internal deployment at Jane Street was delayed by 2.5 years due to the need to forward-port features added to Runtime 4 and resolve significant performance regressions.
- Initial benchmarks of Runtime 5 revealed performance degradations of 10–20% and memory usage increases of 10–21%, far exceeding the acceptable 3% loss projected in the design paper.
- A "Mark Delay" patch was implemented to address "floating garbage," adding an explicit synchronization point to allow marking and sweeping to complete in fewer than two cycles, reducing average object lifetime delays.
- The original pacing algorithm for off-heap memory relied on a flawed closed-loop control system where the parameter $H$ (heap size) created oscillations and inaccurate collection triggers.
- Jane Street redesigned the pacing logic into a unified open-loop steady-state model where a constant number of bytes are collected per word allocated, canceling out dependencies on heap state.
- The team identified "load-bearing bugs" where production systems had adapted to incorrect behaviors in Runtime 4; fixing these bugs initially caused further regressions before stability was restored.
- Backporting Runtime 5 to language versions that still supported Runtime 4 was necessary to enable valid for-for-like benchmarking and mitigate the risk of breaking production workflows.
- Jane Street is releasing "OxCamel" (OCaml Oxidized), a production-ready branch containing these parallel features, intended for researcher experimentation rather than general community stability.
- The "Data Race Freedom a la Mode" paper (POPL 2025) introduces type-level guarantees to ensure programs are free of data races, requiring significant refactoring of existing libraries and codebases.
- Jane Street rejects the "pay up front" ownership model of Rust in favor of OCaml's gradual control, which allows developers to access stack allocation and linear types only when necessary while retaining GC safety.
- The company explicitly avoids building concurrent collectors for sub-millisecond latency trading systems, preferring to use zero-allocation code or FPGAs where determinism is critical.
- The decision to maintain and evolve OCaml rather than adopt a new language was driven by the massive sunk cost of 67 million lines of code and the strategic advantage of attracting high-quality talent through a niche language.
- Jane Street utilizes "Magic Trace" and Intel Processor Trace for deep performance analysis, capturing branch execution with nanosecond granularity to diagnose GC pacing and contention issues.
- Future work includes "heartbeat scheduling" to dynamically adapt parallelism and tools to help developers understand why parallel programs underperform due to cache-line contention or scheduling overhead.