Programmable Hardware with Andy Ray

Hardware vs. Software: Advantages and Trade-offs

• Hardware allows for custom architectures optimized for specific problems, enabling fine-grained trade-offs between performance, cost, and power usage that are impossible with general-purpose CPUs.
• The primary advantage of hardware is the ability to create hybrid designs combining CPUs, digital signal processors (DSPs), and custom blocks, whereas software is often limited by the fixed architecture of the underlying CPU.
• Hardware design faces significant downsides compared to software: it is more difficult to write, requires reams of support libraries, and lacks a vast, mature open-source ecosystem.
• The hardware ecosystem relies on Intellectual Property (IP) suppliers—entities selling pre-designed circuit bundles for integration—which is a vastly smaller market than the software library community.
• Hardware development is characterized by a "paradox of complexity": while individual logic gates are simple, managing the interactions of billions of simple components in a large design is enormously difficult.
• Industry-standard abstraction relies on synthesizers that convert high-level components (adders, multiplexers) into low-level NAND/NOR gates, though early designs (like the first ARM processor) were sometimes drawn at the transistor level.
• Hardware designs typically offer order-of-magnitude improvements in bulk throughput and latency compared to equivalent software implementations.
  • In video coding, hardware FPGAs could achieve real-time 1080p with full feature sets using a fraction of the power required by software on modern x86 processors.
• Hardware provides superior determinism, with latency variance often staying within 20 nanoseconds across the 10th to 99th percentile, whereas software latency frequently spikes into microseconds due to non-determinism.
• Modern CPUs introduce non-determinism through parallel execution heuristics, speculation, and prefetching, making performance prediction difficult.
• Hardware is inherently deterministic at the clock-cycle level, though non-determinism can be introduced via DDR memory access rules (banking, refresh cycles) or interactions with external systems.
• Unlike software, hardware designs must be verified to handle "line rate" traffic; once the physical layout is complete, the system consumes data at a predictable, guaranteed maximum throughput.

FPGA vs. ASIC: Reconfigurability and Economics

• Field Programmable Gate Arrays (FPGAs) consist of a matrix of Lookup Tables (LUTs) and programmable routing, allowing the chip to be reconfigured to perform different functions without physical changes.
• Application-Specific Integrated Circuits (ASICs) are custom-fabricated chips designed for a single function, requiring a foundry to physically etch the design onto silicon.
• FPGA reconfigurability allows for rapid iteration: new designs can be deployed in the field within days, whereas ASICs require weeks to months for fabrication and risk obsolescence if bugs are found post-production.
• ASICs outperform FPGAs in every performance metric: power consumption is 3–10x lower, physical area usage is 10x smaller, and operating frequencies are significantly higher.
• The economics of ASICs only become favorable at massive production volumes (e.g., tens of millions of units), whereas FPGAs remain the superior choice for smaller batches despite higher per-unit costs.
• A single high-end FPGA can cost approximately $8,000 for a small batch, but bulk contracts with manufacturers can reduce that price to ~$500, reflecting the cost of reserved foundry time and storage.

HardCAML: A Modern Hardware Design Language

• Andy Ray developed HardCAML out of frustration with Verilog's lack of a robust software core, which made writing and maintaining complex test benches difficult.
• HardCAML is an embedded domain-specific language (EDSL) built within OCaml, allowing hardware designs to be constructed as functions in a powerful, general-purpose programming language.
• The primary benefit of HardCAML is its integration with software toolchains: it enables instant simulation, waveform viewing, and the use of OCaml's "expect test" framework for continuous integration.
• HardCAML produces a "graph" of hardware components (adders, multiplexers) that is passed to synthesizers, ensuring that the design logic is decoupled from the low-level wiring details.
• The workflow mimics modern software development, leveraging build systems, editor integration, and automated testing to catch errors immediately, rather than relying on expensive, GUI-heavy traditional EDA tools.
• HardCAML maintains an abstraction level similar to Verilog (focusing on logic gates and timing) but leverages OCaml's composability and modularity to manage design complexity.
• While alternatives like BlueSpec offer higher-level abstractions (atomic actions that auto-wire modules), HardCAML preserves precise control over signal timing, which is critical for tasks like DDR memory interfacing.
• Unlike High-Level Synthesis (which converts serial C code to parallel hardware), HardCAML allows designers to explicitly write parallel logic that maps directly to the hardware's nature.

Specific Applications in Financial Trading

• Jane Street applies hardware design primarily to high-speed network packet processing, utilizing FPGAs on custom network interface cards (NICs).
• Custom hardware can parse and filter market data protocols beyond standard IP/TCP layers, inspecting the specific data payload of exchange feeds.
• Hardware pre-processing offloads work from the host CPU by splitting, filtering, or reconstructing packets before they enter the software stack, reducing bus load and memory usage.
• In volatile markets, trading systems must consume several million messages per second; hardware ensures deterministic processing of this load (up to 100 Gbps) without the risk of software "falling over."
• Hardware designs allow for parallel processing of multiple data sources (e.g., eight exchanges simultaneously) with line-rate guarantees that software systems cannot consistently provide during peak loads.

Open Source and Future Aspirations

• HardCAML is an open-source project, but the community adoption remains low compared to competitors like Chisel (based on Scala) due to a lack of public, realistic example designs.
• Jane Street is releasing new HardCAML libraries to the public to facilitate learning and adoption, aiming to bridge the gap between internal tooling and external usability.
• The project hopes to eventually release more complex, real-world designs, though this is complicated by the need to separate proprietary code from general-purpose IP.
• The speaker notes that hardware innovation often lags behind software in adoption, citing the 40-year gap between the invention of garbage collection and its mainstream use in Java.
• Future trends suggest a move toward "software-defined" management for hardware and networks, applying standard software engineering rigor (CI/CD, code review) to hardware development flows.