Matt Godbolt: Advanced Skylake Deep Dive

Speaker & Context

• Matt Godbolt, a C++ developer known for Compiler Explorer (internally named "Godbolt"), emulators, and reverse engineering CPU microarchitecture.
• Transitioning from a one-year non-compete to a new role at HRT.
• Focus of talk: Skylake-era (2019–2020) microarchitecture, acknowledging limitations due to lack of access to modern server CPUs (e.g., Granite Rapids) due to rental costs.
• Reliance on community reverse engineering (e.g., Agner Fog, Travis Downs, Andreas Abel) rather than official Intel documentation for many internal details.

CPU Pipeline Overview

• Front End: Handles Fetch, Pre-decode, Decode, and Renaming; responsible for converting variable-length x86 instructions into fixed-length micro-operations (µops).
• Back End: Handles Out-of-Order (OoO) execution, scheduling, execution units, and Write-Back; decoupled from program order until retirement.
• Retirement: Ensures results are committed to the architectural state in the exact program order, effectively "un-doing" OoO reordering for the programmer.

Front End Mechanics (Skylake Specific)

• Instruction Fetch:
  • Fetches 16 bytes at a time based on branch prediction.
  • Wasted bandwidth occurs if a branch lands on a non-16-byte boundary.
• Pre-Decode:
  • Uses heuristics to identify instruction boundaries within 16 bytes.
  • Can handle up to 5 instructions per cycle.
  • Performs Macro-Fusion: Combines Compare + Branch or Arithmetic/Logic + Branch into a single internal operation to improve throughput.
  • Limitations: "Length changing prefixes" can confuse the pre-decoder, incurring a ~3-cycle penalty.
• Decode:
  • Consists of four decoder units.
  • Decoder 0 is the "legacy" decoder capable of handling complex instructions (up to 4 µops) and routing to the microcode sequencer.
  • Decoders 1–3 handle simple instructions (1 µop each).
  • Micro-Fusion: Combines instructions with memory operands (e.g., ADD [mem]) into a single µop in the queue, which later splits into a load and an ALU op.
• Micro-Op Cache (M-Cache):
  • Stores decoded µops to bypass the legacy decode pipeline on jumps.
  • Delivers 4–6 µops per cycle (measured 4 in Skylake).
  • Constraints: 32 sets x 8 ways; max 3 ways allowed per 32-byte block to prevent cache thrashing.
  • Branches (even not-taken) end cache lines, creating boundaries for the cache.
• Loop Stream Detector (LSD):
  • Detects loops already in the buffer and replays them without fetching/decoding.
  • Can unroll loops up to 8x on Skylake.
  • Critical Issue: Disabled on Skylake due to a "nightmare" bug in Debian involving the OCaml runtime's use of 16-bit register high/low parts, causing unpredictable behavior; Intel released a microcode patch to disable the feature.

Renaming & Physical Registers

• Function: Breaks false dependencies (WAR/WAW) by mapping architectural registers (EAX, RDI) to a vast pool of physical registers (hundreds).
• Data Structures:
  • Physical Register File: Stores actual values; broken into different widths (e.g., 64-bit for GPRs, 512-bit for AVX).
  • Register Alias Table (RAT): Maps architectural names to physical register IDs.
  • Reorder Buffer (ROB): Tracks instruction state in program order (224 entries on Skylake); stores source/destination info for dependency resolution and undoing mispredictions.
  • Reservation Station (Scheduler): Holds ready µops waiting for execution units; entries are not in program order.
• Elimination Optimizations:
  • XOR Elimination: XOR EAX, EAX renames to a hardcoded "zero" physical register; no execution unit usage.
  • Move Elimination: MOV RBX, RAX renames RAX to point to RBX's physical register; no execution unit usage.
    • Alias Limit: Supports 4 simultaneous aliases per physical register; exceeding this locks the register until overwrites occur.
  • Arithmetic Elimination: Small increments/decrements may be tracked as offsets in the RAT rather than executed immediately.
    • Side Effect: Shifts depending on eliminated arithmetic registers may incur an extra cycle (2 cycles total) due to barrel shifter setup timing.

Back End & Execution

• Scheduling:
  • Scheduler picks ready µops based on operand availability and port congestion.
  • Port Allocation Strategy: Tends to assign operations to the highest available port number (most limited) to preserve lower ports for more versatile tasks.
• Execution Ports (Skylake):
  • ALU Ports (0, 1, 2, 3): Handle integer arithmetic, logic, shifts, and permutes.
  • Load Ports (4, 6): Handle address generation and memory loads.
  • Store Address Port (5): Handles store address generation only.
  • Store Data Port (7): Handles store data write-back.
  • Latency Balancing: Hardware attempts to balance latencies (e.g., 3-cycle and 5-cycle ops) to prevent port contention.
• Memory Order Buffer (MOB):
  • Store Buffer: Holds speculative stores; handles forwarding to subsequent loads (Store-to-Load forwarding).
  • Load Buffer: Tracks loads and predicts aliasing to avoid unnecessary waits.
  • Store Color: A mechanism to identify which previous stores could affect a load; helps in determining if a load can proceed speculatively.
  • Fencing: LFENCE/SFENCE operations drain buffers to ensure ordering.
• Floating Point & Exceptions:
  • Denormals: Can trigger a pipeline flush to microcode for handling very small numbers (deoptimization).
  • Retirement: Commits results, frees physical registers, and marks stores as "retired" in the MOB for eventual commit to memory.

Reverse Engineering Methodology

• Technique: Relies on performance counters, microcode tracing (e.g., Intel's "Eureka"), and precise timing measurements.
• Tools: Agner Fog's manuals, Travis Downs' UARCHBench, and custom timing loops to detect pipeline stages.
• Challenges: Many internal details (e.g., specific RAT algorithms, exact ROB structure) are not in public manuals; often inferred from side-channel effects or counter behavior.
• Intel Transparency: Intel provides limited information (Pink/Yellow books) to high-volume customers; public docs often omit details to avoid "locking" themselves into specific behaviors (e.g., Meltdown/Spectre side-channels revealed via reverse engineering).

Key Insights & Future Outlook

• Register Count: Modern CPUs have hundreds of physical registers; increasing architectural register count (as requested by some compilers) yields diminishing returns as the renamer handles the rest.
• Branch Prediction: A major differentiator between vendors; reverse engineering has revealed flaws (e.g., bit-5 direct mapping in Skylake) that can be exploited for side-channels or performance tuning.
• Code Quality: Simple, small loops aligned on 16-byte boundaries perform best; complex instructions (divides, legacy prefixes) should be avoided.
• ISA Impact: While ARM has simpler fixed-length instructions, modern high-performance ARM chips utilize similarly complex decode and prediction front-ends, narrowing the performance gap.
• Future Work: Community continues to reverse engineer newer architectures (e.g., Intel's newer branch predictors, AMD's different approaches) using the same timing and counter-based methods.