Patel explains Inference X was created because vendor-claimed performance is unattainable with any available framework.
“There's no way to get the performance that vendors like to claim. And so our whole thing there was, well, how do we actually have a benchmark that represents what people actually get on performance?”
Patel says VLLM and SGLANG now use Inference X as their competition leaderboard instead of Twitter posts.
“They used to post on Twitter all the time about how they had beat the other one in certain some kind of performance and now they use Inference X all the time.”