newsfilter.io
Lecture, Conference Presentation

Why OCaml

Technical Motivations and Language Characteristics

  • Language Selection: Jane Street adopted OCaml, a functional, statically typed language, as its primary development tool for almost all systems approximately a decade ago.
  • Performance Trade-offs: The firm chose a static functional language to achieve compiled-language performance (comparable to C/C++ and Java) without the syntactic verbosity typically associated with those languages.
  • Type Systems and Safety:
    • OCaml utilizes powerful type inference that allows the compiler to deduce types (e.g., int list -> int) without explicit annotations, balancing safety with concision.
    • The type system enforces "exhaustiveness checking" on pattern matches, preventing runtime errors caused by unhandled cases (e.g., missing a null check in dictionary lookups).
    • The "Option" type mechanism forces developers to explicitly handle the absence of a value, eliminating a class of bugs common in dynamically typed languages like Python.
  • Immutability and Composition:
    • The language enforces a functional style where variables do not mutate, reducing "side effects" and implicit communication channels between code modules.
    • Immutability significantly simplifies reasoning about code composition and makes concurrency safer by removing race conditions inherent in mutable data structures.
  • Development Workflow:
    • The firm utilizes a Read-Eval-Print Loop (REPL) for rapid experimentation, a feature often absent in heavy compiled languages like C++ or Java.
    • The language supports high-level abstractions like first-class functions and algebraic data types, allowing complex logic (e.g., boolean expression evaluators) to be written with significantly less boilerplate than in Java.

Organizational and Strategic Decisions

  • Hiring Strategy: Using an obscure language acts as a signaling mechanism to attract highly motivated and skilled candidates who possess a genuine interest in functional programming, bypassing the large, less-selective pool of generalist Java developers.
  • Talent Acquisition Metrics: An early hiring campaign via the OCaml mailing list yielded a high conversion rate, with 12 of 15 initial responses deemed worth interviewing and 3 of 5 final interviewees hired as high-performing employees.
  • Training Implementation: The firm mandates a one-month intensive boot camp for traders to learn OCaml, resulting in approximately 50% of participants reaching a level of competence sufficient for productive work.
  • Academic Integration: The choice of OCaml aligns with curricula at institutions like MIT and Harvard, where OCaml is a standard teaching language, facilitating internships and recruitment from top universities.
  • Codebase Consolidation: A single language is used for all tasks ranging from small automation scripts to multi-billion-dollar trading systems, enabling knowledge sharing and code reuse across the entire 70+ developer organization.
  • Hiring Paradox: While the language limits the total candidate pool, it effectively filters for individuals with specific technical interests, turning a potential disadvantage (smaller pool) into a selective advantage (higher quality candidates).

Challenges and Industry Trends

  • Ecosystem Limitations: The firm acknowledges significant friction due to the lack of third-party libraries and mature tooling compared to mainstream languages, particularly for web development tasks.
  • Tooling Improvements: The firm actively contributes to the community to resolve tooling gaps, supporting the development of Opam (package management), Merlin (IDE integration), and OCP Indents (auto-formatting).
  • Institutional Inertia: The speaker notes that the programming language industry moves irrationally slowly, with major innovations like garbage collection taking roughly 35 years to reach mainstream adoption.
  • Adoption Barriers: Switching costs are immense due to existing large codebases (millions of lines in OCaml) and the deep emotional attachment developers form with their existing toolchains.
  • Industry Usage: While rare among major firms, OCaml is used by Facebook (for Hack and Flow compilers), Bloomberg, and to a lesser extent by Twitter (using the related language Scala) and Haskell.
  • Testing vs. Verification: Due to the "tail-heavy" nature of financial markets and the presence of adversarial trading environments, the firm prioritizes the universal guarantees provided by static type systems over empirical testing, which may miss rare, high-impact events.
  • Future Outlook: Functional, statically typed languages are becoming more widely discussed and taught, though their actual usage in production remains limited compared to dynamic or imperative languages.

Narrative Arcs and Analogies

  • Spreadsheet Programming: The speaker contrasts spreadsheet programming (where data is visible and code is hidden) with traditional programming to illustrate how functional languages shift the focus toward explicit data transformations and rules.
  • John Carmack's Stance: The argument for immutability is reinforced by citing legendary game developer John Carmack's advocacy for avoiding side effects in C++ to improve code decomposability.
  • The "Brilliant" Language Critique: Addressing Rob Pike's criticism that engineers cannot understand "brilliant" languages, the speaker argues that OCaml is accessible enough for non-researchers, citing successful training results and short internships as evidence.