Conference Presentation, Lecture
Why Functional Programming Doesn't Matter
- Core Thesis: Jane Street argues that while John Hughes' 1988 paper correctly identifies high-order functions, purity, and laziness as key functional features, the primary driver for functional programming's success in their domain is expressive static types, not the other three.
- Jane Street Profile:
- Operates as a proprietary trading firm acting as a market maker on roughly 200 global stock exchanges.
- Executes approximately one million trades daily.
- Processes approximately one petabyte of historical market data.
- Uses OCaml as the primary language for all production systems.
- Primary Engineering Priorities:
- Correctness: Critical because the firm trades its own capital; errors result in immediate job loss and financial liability.
- Clarity: Code must be self-evident to ensure correctness is verifiable by humans, not just the compiler.
- Dexterity: The system requires high agility to deploy new trading strategies or adapt to regulatory changes quickly, necessitating low boilerplate and safe refactoring.
- Performance: Requires reasonable speed for analysis and predictability; OCaml is preferred over Haskell because its performance characteristics are predictable and resistant to subtle regressions from compiler optimization changes.
- Critique of Hughes' Three Features:
- High-Order Functions: Universally agreed upon as useful for defining custom control flow mechanisms and modularizing large functions, though not the top priority.
- Purity: Valued but recognized as difficult to enforce at all scales; strict purity can sometimes introduce unnecessary complexity when dealing with real-world side effects (e.g., writing to exchanges).
- Laziness: Explicitly de-prioritized; the speaker cites unpredictability in execution timing as a risk for high-frequency trading environments where volatile times of day require consistent system performance.
- The Primacy of Expressive Static Types:
- Eliminating Null Pointers: Types like
optionormaybeforce developers to handle absence explicitly, removing "null pointer exceptions" (described as a "billion-dollar mistake" by Tony Hoare) from the codebase. - Compile-Time Enforcement of Logic: When a variant type (e.g., message types like ACK, OUT, FILL, BUST) is extended, the compiler forces updates in all downstream pattern matches, preventing logic gaps in new business rules.
- Excluding Illegal States: Types can be engineered to make invalid states unrepresentable (e.g., distinguishing between left, right, or both values in a merge operation), removing impossible code branches and simplifying code reviews.
- State Encoding: Complex invariants, such as mutually exclusive data fields in connection objects (e.g.,
disconnected_timeonly being valid when the state is disconnected), are enforced by the type structure itself rather than runtime checks or comments.
- Eliminating Null Pointers: Types like
- Future Outlook:
- Jane Street continues to use OCaml as it provides all functional features (including purity and higher-order functions) without requiring a compromise on static type expressiveness.
- If forced to choose a single defining feature for their stack, expressive static types are selected over laziness, purity, or higher-order functions.