Patel argues silicon supply can be optimized for either high throughput or high interactivity use cases.
“Supply of silicon can go many ways. You can either leverage it to high throughput things or high interactivity things.”
Patel says InferenceX now has over $80 million of compute across multiple vendors.
“we have over $80,000,000 of compute GPUs from NVIDIA AMD, TPUs from, Google, Tranium from Amazon, and we run this benchmark constantly on the newest inference engine, newest drivers, newest, PyTorch version,”
Patel claims hardware improved 30x from Hopper to Blackwell for DeepSeek on optimized deployments.
“from Hopper to Blackwell, which is all we've had over the last three years, roughly 30x improvement on DeepSeek, on the most optimized deployment, which you can see on InferenceX there's about a 30x improvement.”
Patel explains Bumpgate issue where thermal expansion differences caused GPU solder ball failures in Apple laptops.
“And what ended up happening is because of that different rate of expansion, the solder balls connecting the chip and the board would crack.”
Patel claims four to five percent of NVIDIA GPUs fail during initial burn-in testing within two weeks.
“Buy a good chip from NVIDIA, and then it ends up failing like constantly, right?”