The Minutes of Dylan Patel on gpu utilization https://minutesof.com/dylan-patel/on/gpu-utilization/ Everything Dylan Patel has said on gpu utilization: 2 verbatim quotes between December 2023 and July 2026, each with a timestamp and a link to the… en Mon, 31 Aug 2026 13:10:13 +0000 Patel argues prefill caching drastically raises GPU utilization by eliminating repeated context recalculation https://minutesof.com/q/1c63392d-c6b8-44b1-8b80-f8d850c63669/ https://minutesof.com/q/1c63392d-c6b8-44b1-8b80-f8d850c63669/ “Now that means your GPU utilization rises drastically, and the amount of time that the GPUs are generating new tokens is far, far higher than the amount of times that GPUs are generating or recalculating the context, which is not necessarily providing any value to your operations.” — RAISE Summit Thu, 16 Jul 2026 16:44:57 +0000 gpu utilization Patel explains model bandwidth utilization is the critical metric for inference, unlike training where MFU mat https://minutesof.com/q/815c0b87-b8a9-42b6-bfc0-c6e3c82677fb/ https://minutesof.com/q/815c0b87-b8a9-42b6-bfc0-c6e3c82677fb/ “But on inference, it's not being talked about much, but model MBU, model bandwidth utilization is the important factor.” — Latent Space Tue, 05 Dec 2023 18:16:28 +0000 gpu utilization