Dylan Patel · Scaling Intelligence · 12 November 2024
“But if you batch higher, k b cache is not just a memory capacity issue, it's also a memory bandwidth issue.”
Verbatim excerpt with a timestamp. The full recording is at the source; we link out and do not host it. Everything Dylan Patel is on record saying.