Conference Presentation
Raph Levien: A Taste of GPU Compute
- Single-core performance is predicted to stagnate due to the halt of Dennard scaling approximately 15 years ago, making increased parallelism the sole method for achieving high throughput on modern chips.
- GPUs are expected to offer roughly an order of magnitude more raw floating-point operations per second and throughput per dollar (and per watt) compared to CPUs across mobile, desktop, and server markets.
- Future chip architectures will dedicate the vast majority of area to GPU execution units, with an estimated count between 24 and 64 units, while CPU cores are treated as a minor addition.
- Hardware-specific execution parameters include Intel SIMD lane widths of 8 or 16, NVIDIA at 32, and AMD at 64, with each thread comprising 8 lanes and a 4KB register file (512 bytes per lane).
- Operational latency is projected at approximately 2 nanoseconds for execution unit operations, 100 nanoseconds for sub-slice operations (grouping eight units on Intel), and 300 nanoseconds for main memory access.
- The optimal thread group size for maximize parallelism is estimated between 32 and 1024 threads, with thread group shared memory capped at 64 kilobytes per thread.
- A two-stage pipeline using tiles is planned to optimize 2D rendering, expected to improve work factor and read bandwidth by a factor of approximately 256 compared to sequential bounding box approaches.
- Functional programming structures are predicted to align closely with machine learning workloads, which currently drive a significant portion of GPU utilization despite 90% of general compute throughput remaining idle in non-specialized scenarios.
- The industry is expected to see a shift where many workloads migrate from CUDA to standardized interfaces like Vulkan and WebGPU, supported by SpireV as a foundation, though WebGPU adoption is predicted to be driven by hardware testing coherence.
- Performance risks include shader divergence causing idle lanes when branches do not align, poor write bandwidth in sequential approaches with overlapping objects, and the difficulty of automatic compiler optimization for thread block layouts without manual tuning.
- Future divergence between graphics and machine learning is anticipated due to specialized tensor hardware, although general tooling and deployment improvements may eventually facilitate convergence.
- The speaker plans to present simplified hardware models to avoid specific chip variations and will exclude mobile GPUs, which are considered sufficiently distinct from desktop architectures.
- Skepticism exists regarding the ability of high-level languages and smart compilers to automatically map to hardware efficiently, favoring languages that allow explicit hardware mapping with minimum friction.
- Controlling subgroup width via Vulkan extensions is noted as not widely implemented or available on Windows, representing a current limitation in hardware configuration.