Interview, Fireside Chat, Webinar
Clock Synchronization with Chris Perl
- Hardware timestamping on network interface cards eliminates scheduling and interrupt delays, enabling systems to achieve error ranges of approximately 20 microseconds, compared to 35 to 40 microseconds for systems lacking this capability.
- Current optimized configurations can attest to errors no worse than 35 microseconds, while the default PTP protocol sends synchronization messages every second, exceeding the standard NTPD setting of once every eight seconds but remaining adjustable on daemons like Chrony to 16 times per second.
- Existing systems at Jane Street have experienced initial drifts of a couple hundred microseconds and occasional disturbances causing offsets of 400 to 500 microseconds due to CPU heat profile changes, which can cause oscillator frequency drift of 50 to 100 microseconds per second.
- European financial regulations (MIFID 2) currently require UTC synchronization within 100 microseconds, a threshold that clients can meet using local NTP servers within 100 microseconds to avoid adding the 600-microsecond error of distant servers, while sampling data every 10 seconds is deemed sufficient to detect persistent compliance violations.
- Hybrid architectures are proposed where GPS appliances distribute time via PTP to Linux servers, which then serve NTP to the rest of the network using hardware timestamping and interleaved modes to leverage precision similar to PTP without needing a dedicated time synchronization network.
- White Rabbit, which combines PTP with synchronous Ethernet to synchronize frequencies, is identified as necessary for sub-nanosecond synchronization, and future hypothetical regulations requiring 10-microsecond accuracy ("MIFID 3") would likely necessitate this technology.
- Moving to sub-nanosecond or 10-microsecond precision levels requires significant re-engineering of reliability and redundancy models to address potential black-box failures in GPS appliances, as current PTP models lack robust false-ticker detection and may propagate "crazy" timestamps from faulty grandmasters.
- Standard PTP behavior using multicast delay requests is described as pathological, leading to a preference for hybrid unicast models to avoid quadratic message complexity, while the Best Master Clock Algorithm may select a clock as "best" even if it reports incorrect time due to misconfiguration.
- Network delay asymmetry remains a critical error source, as applications generally suffer from backward time jumps, prompting time daemons to prefer "slewing" over "stepping" to correct drift, though older Windows NTP daemons historically used stepping which created sawtooth patterns.
- Newer time daemons like Chrony can remove one second of error in 12 seconds, managing daily drifts effectively, whereas older desktop computers or systems with poor oscillators may experience corrections worse than 50 parts per million.