Module 4 of 7
GPU performance
How GPU code is written and made fast: the CUDA programming model, coalesced access and occupancy, tiling, fusion and IO-aware kernels, and reading a profiler.
The architecture module said what the hardware can do. This module is about getting a kernel close to it. It starts with how work is mapped onto grids, blocks, and threads, then covers the patterns that decide most kernel performance: coalesced memory access, enough resident warps, tiling for reuse, and fusing operations so intermediate results never touch HBM. It ends with reading a profiler report to find which of these is the problem.
Each lesson counts the bytes and transactions with a short Python program, so you can check a kernel design on paper before writing it.
Lessons
- 4.aThe programming modelHow does one kernel function turn into millions of threads, and how does each thread know which data is its own?8 min
- 4.bCoalescing and occupancyWhy does the same amount of data cost eight times more when accessed with a stride, and what decides how many warps an SM can hold?8 min
- 4.cTiling a matmulHow does a matmul kernel turn a memory-bound loop into one that keeps tensor cores busy?9 min
- 4.dFusion and IO-aware kernelsWhy does combining kernels make a model faster without doing less arithmetic, and how does FlashAttention avoid the quadratic memory traffic of attention?9 min
- 4.eProfiling GPU codeGiven a profiler report, how do you tell whether a kernel is limited by compute, memory, latency, or launch overhead?8 min