Patel claims Nvidia achieves 50% sustained performance on common 8k gemm while AMD gets only 30%.
“if you just take a eight k by eight k by eight k gem, very common shape and you run it on the map mode unit of AMD and Nvidia, you get like 50% sustained performance on Nvidia and you get like 30% sustained performance on AMD.”
Patel says H100 cannot physically reach advertised 2,000 teraflops, maxing at 1,300 teraflops for matrix operations only.
“In fact, it's physically impossible to get 2,000 teraflops out of the H100. Even though the advertises it, at most you can get 1,300 FP8 floating point eight teraflops.”
Patel quantifies Hopper performance improved 30-40% over 2024 with Blackwell showing similar gains in months.
“Even Hopper in January 24 to December 2024, you still had like a 30%, 40% performance improvement. And likewise for Blackwell, you've seen a similar sort of improvement, right?”
Patel says GPUs in a 16,000 GPU cluster show up to 10% speed variation from manufacturing differences.
“But even within a cluster of like 16 k GPUs, GPUs will be slower by up to 10%. Right? So there there's a lot of variation even in the manufacturing.”