What does the ridge point mean?

The ridge point is the arithmetic intensity where the bandwidth roof meets the compute roof. Below it, the maximum possible throughput rises with intensity because memory bandwidth is the limiting resource. Above it, the maximum possible throughput is capped by compute peak. It is the dividing line between bandwidth-bound and compute-bound behaviour in the simplified model.

Does being bandwidth-bound mean the kernel is badly written?

Not necessarily. Some workloads inherently move many bytes per FLOP, such as single-token decode that streams model weights or KV cache data with little reuse. Optimisation may still reduce bytes, improve locality, or batch work, but a bandwidth-bound point can be the expected consequence of the algorithm and serving shape rather than a simple implementation bug.

Why can prefill and decode sit in different places on the same roofline?

Prefill processes many prompt tokens together, so weight data can be reused across a larger block of computation. That raises arithmetic intensity and can make matrix multiplication approach the compute roof. Decode often handles one new token at a time, so it repeatedly reads large weight and KV data streams for comparatively little arithmetic, placing it near the bandwidth roof.

When does a roofline analysis mislead?

It misleads when the real bottleneck is outside the two ceilings being plotted. Latency targets, kernel launch overhead, poor occupancy, cache misses, communication collectives, KV capacity, or page-table indirection can dominate. The honest answer is: it depends on the measured bytes, the sustained hardware limits, and whether the optimisation introduces a different bottleneck.

Roofline

A roofline is a performance model and plot that bounds a kernel by the lesser of the machine’s compute peak and its memory-bandwidth-limited throughput. It relates achieved FLOP/s to arithmetic intensity, so an engineer can see whether more reuse, fewer bytes, or more compute capacity is likely to help.

The problem roofline addresses is that low utilisation is ambiguous. A GPU kernel may run far below peak FLOP/s because the arithmetic units are idle waiting for HBM, or because the arithmetic units themselves are the limit. Looking only at elapsed time or theoretical peak does not tell you which. Roofline gives a first-order diagnostic by comparing the work a kernel performs with the data it must move.

To draw it, put arithmetic intensity, FLOPs per byte moved, on the x-axis and achieved throughput on the y-axis, usually on log-log axes. The sloped line is memory bandwidth multiplied by intensity. The flat line is peak compute throughput. Their intersection is the ridge point, equal to compute peak divided by bandwidth peak. A kernel plotted left of the ridge is bandwidth-bound; right of it, compute-bound.

The model is deliberately incomplete. It assumes the relevant bandwidth and compute ceilings are known and sustained, and it hides launch overheads, cache effects, synchronisation, interconnect traffic, occupancy, and latency constraints. It also depends on what you count as bytes moved: HBM traffic, cache traffic, or logical tensor traffic can place the same kernel differently. A roofline is a bound and a compass, not a benchmark.

Engineers meet rooflines when reasoning about GPU kernels, especially inference workloads. Single-token decode for a dense Transformer often has low arithmetic intensity because weights and KV cache bytes are streamed for little reuse. Prefill can move rightward because the same weights are reused across many tokens in matrix-matrix work. FlashAttention, batching, paged KV caches, and speculative decoding can all be read as attempts to move points or expose a different bottleneck.

Go deeper

Inference EngineeringThe RooflineDraw and read a roofline plot. Place a kernel on it, identify the ridge point, and determine which side of the ridge it sits on.Open the lesson

Common questions

What does the ridge point mean?: The ridge point is the arithmetic intensity where the bandwidth roof meets the compute roof. Below it, the maximum possible throughput rises with intensity because memory bandwidth is the limiting resource. Above it, the maximum possible throughput is capped by compute peak. It is the dividing line between bandwidth-bound and compute-bound behaviour in the simplified model.
Does being bandwidth-bound mean the kernel is badly written?: Not necessarily. Some workloads inherently move many bytes per FLOP, such as single-token decode that streams model weights or KV cache data with little reuse. Optimisation may still reduce bytes, improve locality, or batch work, but a bandwidth-bound point can be the expected consequence of the algorithm and serving shape rather than a simple implementation bug.
Why can prefill and decode sit in different places on the same roofline?: Prefill processes many prompt tokens together, so weight data can be reused across a larger block of computation. That raises arithmetic intensity and can make matrix multiplication approach the compute roof. Decode often handles one new token at a time, so it repeatedly reads large weight and KV data streams for comparatively little arithmetic, placing it near the bandwidth roof.
When does a roofline analysis mislead?: It misleads when the real bottleneck is outside the two ceilings being plotted. Latency targets, kernel launch overhead, poor occupancy, cache misses, communication collectives, KV capacity, or page-table indirection can dominate. The honest answer is: it depends on the measured bytes, the sustained hardware limits, and whether the optimisation introduces a different bottleneck.

Every term