Patel calculates o1 models increase inference costs by roughly 50x compared to standard models per query.
“Cost increase for a single token to be generated is four to five x, but then I'm generating 10 x as many tokens.”
Patel says DeepSeek's model is 400 to 600x cheaper than original GPT-4.
“And so now DeepSeek's model is something like 400 to 600x cheaper than the original GPT-four.”
Patel states each gigawatt AI deployment costs tens of billions of dollars end-to-end.
“Each of these gigawatt deployments once you take it from all the way from the power to the GPUs and the cloud contract, those are tens of billions of dollars each gigawatt.”
Patel reports that tokens now represent 30% of his 90-person company's costs versus 70% for employees.
“My my own company of 90 people, 30% of my cost now is tokens versus 70% employee costs.”