Patel predicts five to seven companies will train GPT-4 scale models in the next year.
“I do believe that there's going to be five to seven companies that will have a GPT-four size model, at least, right, in terms of total flops.”
Patel predicts open source will match GPT-4 but frontier models will continue advancing beyond it.
“Open source will match GPT-four, but then it's like, what about GPT-four Vision? Or what about five and six and all these kind of stuff?”
Patel argues training costs are irrelevant, noting GPT-4 used 20,000 A100s.
“training costs are irrelevant. Right? Like, GPT four, right, like, 20,000 a one hundreds, that's that's like, I know it sounds like a lot of money.”
Patel estimates 10 companies will have enough compute to beat GPT-4 within six months, requiring about 7,000 H100s.
“there's like 10 companies that have enough compute in one single data center to be able to beat GPT-four. Right? Like straight up. Like, if not today, within the next six months.”
Patel says GPT-4 cost hundreds of millions to train but generates billions in revenue for Microsoft.
“Hundreds of millions of dollars to train g p d four. Yep. And it's generating billions of dollars of revenue. So when you say, like, hey.”