Lecture, Keynote, Presentation
How to Build an Exchange
- The system architecture anticipates a market composition of approximately 50% ad orders, 30-40% cancel orders, and 1-2% executions, potentially sorting transactions by price and time while facing peak rates of roughly three million messages per second for the US equities market.
- Processing expectations include maintaining several million live orders across around 10,000 symbols, with participants likely focusing on performance advantages gained over the past decade and requiring the system to remain robust against badly behaved or careless clients.
- Fairness will be prioritized by ensuring simultaneous information delivery via multicast (UDP) to market data, drop ports, and trade reporters, alongside mechanisms to provide price improvements to aggressive orders.
- The core design relies on a single application matching engine running on a single x86 machine, with logic such as scheduled cancellations factored out and auction processes running independently before feeding results back to the engine.
- To handle potential hardware or software failures, the system incorporates retransmitters for message drops, secondary passive matching engines for state rebuilding, and a failover process where an operator manually switches to the secondary engine if the primary crashes.
- Recovery objectives target a rebuilding time of 30 seconds to a minute for application state, utilizing state machine replication and replay capabilities for testability while explicitly avoiding consensus algorithms like Paxos to minimize latency.
- Performance engineering aims for low single-digit microsecond latency using commodity hardware and safe languages like C++, Java, and OCaml, while avoiding deferreds and excessive allocations in the critical path to ensure deterministic behavior.
- System parallelism will likely be limited to the symbol level to enforce atomic risk limits, with unique topics assigned per message to manage concurrency via independent locks, though instrument-level parallelization remains unclear for all order types.
- Protocol mechanics include one-in-flight transaction semantics to simplify rollbacks, sequenced cancel rejects to maintain deterministic state, and specific handling where ports proposing completed transactions must yield to the new state.
- Regulatory compliance is supported by ensuring determinism allows auditors to verify exactly what the system knew at every decision point, with mechanisms to regenerate missing messages if dual writes fail.
- Resilience features include the ability to rebuild open orders on different machines upon hardware port failure and the provision of distinct ports or sequence number spaces if the system replicates at the symbol level to prevent state injection.
- Uncertainty remains regarding whether the system can always guarantee clear parallelization at the instrument level for every order type and situation, though the overall design is expected to scale naturally.