Interview, Fireside Chat
Memory Management with Stephen Dolan
Jane StreetStephen Dolan, KC Sivaramakrishnan, Ron Minsky, Mark Mandelmann, Simon Hughes, Matt Sullivan, Mark Blyther, Paul Lewisohn, Simon Verstraeteker
- The OCaml multi-core garbage collector is imminent and will enable concurrent allocation and access across threads, resolving single-lock limitations while operating incrementally in slices to minimize pauses for soft real-time applications.
- Future garbage collector tuning will allow a single parameter to trade memory usage for speed, with prefetching optimizations overlapping 10 to 30 memory requests to approach a reasonable fraction of raw memory bandwidth.
- The generational hypothesis is expected to remain robust, where minor collections eliminate approximately 90% of allocations, leaving the major collector responsible for long-lived objects.
- The "local types" feature will facilitate safe stack allocation without requiring explicit annotations or heavy machinery, allowing smart constructors to return values that outlive the function frame while preventing dangling references through compiler guarantees.
- "Unboxed types" will represent atomic data and collections directly in registers to avoid heap allocations, enabling C-style record inlining, unboxed variants, and the tight packing of data without current 32-bit integer representation verbosity.
- A new kind system for unboxed types will track data layouts, permitting polymorphic functions only over types with specific compatible layouts, such as those fitting in a single register.
- The unboxed types system will address the representation of
Optiontypes via a nullable kind that prevents ambiguity, while enforcing constraints to avoid nesting nullable types. - Implementation will require the garbage collector to handle densely packed structures with mixed half-pointer and half-data layouts, lifting current restrictions on these configurations.
- Performance improvements include multi-argument returns without intermediate heap allocations, mutation primitives for unboxed structures, and efficient use of data structures parameterized by types that do not require garbage collector support.
- Smart constructors for local types will reuse exact stack memory for subsequent allocations, improving L1 cache behavior compared to the minor heap, without requiring programmers to manipulate stack frames directly.
- The system will avoid the "massive stack" issue associated with automatic stack inference and will not be the most flexible option for all cases, though it covers common reference-passing scenarios better than Rust's lifetime annotations in OCaml.
- Unboxed tuples will exist as ethereal immutable structures that can be boxed for heap storage, treated as multiple local variables in separate registers, and used in generic code without generating numerous function copies.
- The OCaml compiler will maintain support for separate compilation, avoiding the necessity of full program recompilation or exclusive reliance on JIT monomorphization for generic functions.
- The runtime will continue to rely on pointer tagging conventions to distinguish integers from pointers, while the ecosystem evolves toward idioms compatible with these specific memory management trade-offs.
- Exposing software prefetching instructions more directly to programmers remains a goal, though it is noted as difficult due to timing sensitivities, and unboxed variant support is currently a work in progress.
- Compiler backend changes may be required to handle return layouts that depend on specific variant cases, and the design includes mechanisms to prevent the nesting of nullable abstract types by default.