Patel claims four to five percent of NVIDIA GPUs fail during initial burn-in testing within two weeks.
“Buy a good chip from NVIDIA, and then it ends up failing like constantly, right?”
Patel explains Bumpgate issue where thermal expansion differences caused GPU solder ball failures in Apple laptops.
“And what ended up happening is because of that different rate of expansion, the solder balls connecting the chip and the board would crack.”
Patel explains AI chip startups like Groq and SambaNova optimized for past model architectures and lost when models evolved.
“And so, like, you look at, like, Grox or Rebis, SambaNova, they all, like, sort of over indexed to the models that were leading at the time when they designed their chips.”
Patel lists six current GPU types benchmarked with NVIDIA B300/GB300, Google TPU and Amazon Trainium coming soon.
“And soon, NVIDIA will be contributing B300, GB300 to us so that we can run this benchmark nightly, as well as some other clouds.”
Patel claims hardware improved 30x from Hopper to Blackwell for DeepSeek on optimized deployments.
“from Hopper to Blackwell, which is all we've had over the last three years, roughly 30x improvement on DeepSeek, on the most optimized deployment, which you can see on InferenceX there's about a 30x improvement.”
“AMD's announcing it in a few weeks and it's very exciting. It's 72 GPUs. It's got more memory than Vera Rubin. It's got more more, memory, flops.”
Patel reveals SemiAnalysis has over 80 million dollars of donated compute from Nvidia, AMD, Google, and Amazon for benchmarking.
“So what we what we have is we have over $80,000,000 of compute GPUs from NVIDIA AMD, TPUs from, Google, Tranium from Amazon, and we run this benchmark constantly on the newest inference engine, newest drivers, newest, PyTorch version, whatever it is.”
Patel says InferenceX now has over $80 million of compute across multiple vendors.
“we have over $80,000,000 of compute GPUs from NVIDIA AMD, TPUs from, Google, Tranium from Amazon, and we run this benchmark constantly on the newest inference engine, newest drivers, newest, PyTorch version,”
Patel argues silicon supply can be optimized for either high throughput or high interactivity use cases.
“Supply of silicon can go many ways. You can either leverage it to high throughput things or high interactivity things.”