Stack Allocation with Locals in OCaml | OCaml Unboxed

Jane Street Local Mode Overview

• Jane Street is developing a "local mode" for the OCaml compiler to optimize memory allocation.
• The implementation exists in an open-source branch currently in development toward being upstreamed into the main OCaml compiler.
• The primary goal is to reduce garbage-collected allocations and eliminate the latency associated with mark-and-sweep garbage collection.

Standard Garbage Collection Mechanics (Baseline)

• In standard OCaml without locals, functions like init allocate cons cells on the global heap.
• A pointer to the newly created list is stored in the list variable within the do it function's stack frame.
• During subsequent loop iterations, new heap allocations overwrite previous pointers, leaving the old list unreachable.
• The runtime identifies unreferenced memory via mark-and-sweep: it traces from stack variables to the heap.
• Memory not reachable from the call stack (like the old list from iteration 3) is reclaimed as garbage.
• Garbage collection forces a "stop-the-world" pause, halting all threads sharing the heap to perform marking and sweeping.
• This process causes significant performance penalties due to cache invalidation and memory traversal latency.
• The technique remains effective for most Jane Street applications but is insufficient for low-latency requirements.

Local Mode Optimization Strategy

• Local mode allocates data on a specialized "local stack" rather than the global heap.
• The local stack is a distinct memory area separate from both the call stack and the heap.
• Values allocated on the local stack are scoped to specific "regions" (e.g., loop bodies) rather than just function calls.
• The core invariant of local mode is that local values "do not escape their region."
• When a region ends (e.g., a loop iteration completes), all memory allocated within that region is known to be unreachable.
• Deallocation in local mode is a constant-time operation involving a single pointer update to reset the local stack pointer.
• This approach eliminates the need for marking, sweeping, cache invalidation, and stop-the-world pauses.
• The mechanism ensures memory reuse for subsequent allocations within the same local stack space.

Architecture and Exclaves

• Local variables must persist across function returns but are released when the enclosing region exits.
• This mismatch between function lifecycles and region lifecycles necessitates storage separate from the call stack.
• "Exclaves" are used to mediate the lifecycle of local values, allowing them to outlive the function that created them without escaping the region.
• This architecture ensures that values remain valid across function boundaries while still enabling efficient, region-based reclamation.