Introduction to Incr_dom: Writing Dynamic Web Apps in OCaml

Project Overview: IncurDOM is a web UI framework developed at Jane Street, primarily for internal trading systems, designed to write efficient, maintainable web interfaces in OCaml.

• Target Use Case: Displays large datasets (e.g., 10,000+ rows) with high-frequency updates (tens of updates per second) where users view and filter data but rarely modify the underlying dataset directly.
• Key Challenge: Maintaining responsiveness and efficiency when sorting, filtering, or editing cells within massive tables without browser lag.

Design Goals:

• Ease of Use: Allow developers to write web UIs with the same ease as standard OCaml applications, avoiding the need for deep JavaScript expertise.
• Maintainability: Enable sharing of business logic between backend (OCaml) and frontend (JS via JS_of_OCaml), ensuring backend changes automatically propagate to the client or trigger compiler errors.
• Performance: Optimize for large data volumes and frequent state changes through partial rendering, virtual DOM diffing, and incremental computations.

Architectural Foundations:

• JS_of_OCaml: A compiler that runs OCaml bytecode in the browser, allowing shared code between server and client and leveraging OCaml's type safety and tooling.
• Elm Architecture: Adopts a strict Model-View-Update pattern:
  • Model: Immutable state of the application.
  • View: A declarative function generating an HTML-like structure based on the model.
  • Actions: Events that trigger state updates; actions are scheduled by the view and applied by apply_action.
• Virtual DOM: Stores a lightweight in-memory copy of the DOM to compute differences (diffing) before patching the actual browser DOM, reducing expensive browser reflows.

Core Optimization Techniques:

• Incremental Computations: Uses the Incremental library to build a dependency graph where only nodes affected by input changes are recomputed.
  • Cutoffs: An optimization where if a node's value remains unchanged despite parent updates, its descendants are not recomputed, saving significant processing time.
  • IncrMap: A specialized library for efficient handling of maps, enabling O(diff size) updates for large datasets (e.g., table rows) rather than O(n) reconstruction.
• Partial Rendering: Renders only rows currently visible in the viewport; off-screen rows are replaced with placeholders to prevent DOM bloat.

Demonstration Workflow:

• Initial State: A simple app with unit model and action types displayed a static header.
• State Introduction: The model was updated to an int list of 100 random counters, displaying a table with row indices and values.
• Styling: Dynamic CSS styles (colors) applied to cells based on value ranges (green <10, blue <100, red >100).
• Interactivity:
  • Single click: Increments a counter.
  • Double click: Resets a counter.
  • Auto-update: Background thread schedules random increments ~20 times/second.
• Performance Scaling:
  • Switching from list to Map improved performance but still required map.data to reconstruct the full list for the table, causing lag at 30,000 rows.
  • Applying IncrMap and incremental view functions significantly smoothed scrolling and updates, though initial load time remained high for large datasets.
  • Added a live "total" sum calculated via unordered_fold on the map, demonstrating efficient real-time aggregation.

Advanced Patterns & Considerations:

• State vs. Model: The model must be immutable and contain all data required for rendering; mutable external state (e.g., server connections) belongs in a separate state module.
• Synchronization: Asynchronous actions require explicit model checks before application to ensure validity against the current state version.
• Focus Management: Custom functions (on_display, update_visibility) are required to keep specific rows (e.g., focused cells) within the viewport after sorting or filtering.
• Partial Rendering Implementation: Required for production-scale apps to handle 10,000+ rows; involves injecting placeholders for non-visible rows and managing scroll offsets.

Comparative Analysis (Q&A Insights):

• Vs. React: IncurDOM lacks explicit component syntax (though composition is natural via the architecture) and relies on OCaml rather than a JS-like syntax.
• Vs. BuckleScript/Reason React: Jane Street prefers JS_of_OCaml over BuckleScript because BuckleScript lags behind the latest OCaml versions, despite BuckleScript producing readable JavaScript.
• Vs. Elm: IncurDOM enforces the Elm architecture more strictly than Elm, which allows flexibility but requires manual pattern adherence. Elm uses commands/subscriptions for side effects; IncurDOM uses the schedule_action mechanism within the view.
• Limitations: Virtual DOM still requires reconstructing the list of children from maps for table rendering, which is a known bottleneck for massive datasets requiring partial rendering.

Future Roadmap & Gaps:

• Components: Explicit component abstractions are currently being developed by the team to improve code reusability.
• Graph Support: Lack of tooling for rendering dynamic graphs; identified as a necessary area for future library expansion.
• Testing: No dedicated testing frameworks yet; current methods rely on console logging and manual browser inspection.
• RPC Best Practices: Recommends polling over persistent WebSocket pipelines to prevent browser crashes during high-frequency updates.

Developer Guidelines:

• Incremental computations should only be used when the overhead of maintaining the dependency graph is justified by the data size or update frequency; small, static models can be treated as constants to avoid unnecessary complexity.
• Asynchronous operations should schedule actions immediately, with validation checks applied in the apply_action phase to handle potential model state drift.