Patel says VLLM and SGLANG now use Inference X as their competition leaderboard instead of Twitter posts.
“They used to post on Twitter all the time about how they had beat the other one in certain some kind of performance and now they use Inference X all the time.”
Patel explains Inference X was created because vendor-claimed performance is unattainable with any available framework.
“There's no way to get the performance that vendors like to claim. And so our whole thing there was, well, how do we actually have a benchmark that represents what people actually get on performance?”
Patel secured over $50 million in donated hardware for InferenceX, expanding to $100 million with TPUs and training.
“We've got over $50,000,000 of hardware donated to us. Once we launch TPUs and training, would actually be over $100,000,000 of hardware.”
Patel reveals SemiAnalysis has over 80 million dollars of donated compute from Nvidia, AMD, Google, and Amazon for benchmarking.
“So what we what we have is we have over $80,000,000 of compute GPUs from NVIDIA AMD, TPUs from, Google, Tranium from Amazon, and we run this benchmark constantly on the newest inference engine, newest drivers, newest, PyTorch version, whatever it is.”
Patel says InferenceX analyzed over $5 million worth of Claude Code production traces.
“we've analyzed over $5,000,000 worth of Claude code traces. Right? So this is real production traffic that people have donated to us,”
Patel says InferenceX now has over $80 million of compute across multiple vendors.
“we have over $80,000,000 of compute GPUs from NVIDIA AMD, TPUs from, Google, Tranium from Amazon, and we run this benchmark constantly on the newest inference engine, newest drivers, newest, PyTorch version,”