Patel predicts open source will match GPT-4 but frontier models will continue advancing beyond it.
“Open source will match GPT-four, but then it's like, what about GPT-four Vision? Or what about five and six and all these kind of stuff?”
Patel identifies distributed training across data centers with lower bandwidth as a key unsolved problem that would unlock massive scaling.
“Everything that we've seen so far is that large scale training has to happen in an individual data center with very high speed networking.”
Patel calculates that multi-trillion parameter models require transmitting 40 terabytes of data every two seconds during training.
“They're doing it for like multi trillion. Right? So let's call it 10,000,000,000,000 parameters, four bytes parameter, that's 40 terabytes of data you need to transmit in two seconds.”
Patel says GPT-4 used 24,000 GPUs, GPT-5 uses 100,000, and Microsoft is building toward a million GPUs.
“GPT four was trained with 24,000 GPUs roughly, and GPT five is on the order of a 100,000. And then they're trying to build this data center over the next few years. That's a million.”
Patel calculates next-generation clusters deliver 15x more compute through five times more GPUs and 3x performance gains.
“So you got you have five x the GPUs, and you have three x the performance per GPU roughly. So then you're at, like, 15 x more compute.”
“Or or that's what it looks like based on or, you know, one gigawatt in twenty twenty six ish. And Meta's Meta's, like, trying to do, like, two gigawatts by early to mid twenty seven.”
Patel says current models use 100,000 GPUs while next generation will require hundreds of thousands or millions.
“And next generation models that are trained on hundreds of thousands or even millions GPUs, right?”
Patel states $10 billion data centers target automated software engineering, not chat models.
“So no one is trying to make with these $10,000,000,000 data centers, they're not trying to make chat models. Right?”
Patel argues that efficiency gains alone without capability increases cannot justify massive AI infrastructure investments.
“That would not pay for all of these build outs. Right? AI is useful today, but it's not capable of doing a lot of things.”
Patel reports Microsoft aims to 10x training capacity every 18-24 months, representing a 10x increase from GPT-5 training.
“We try to 10x the training capacity every eighteen to twenty four months. And so this would be effectively a 10x increase. 10x from what GPD five was trained with.”
Patel estimates Anthropic needs to reach well above five gigawatts of compute capacity by year-end to support revenue growth.
“Anthropic needs to get to well above five gigawatts by the end of this year, and it's gonna be really tough for them to get there, but it's possible.”
Patel predicts OpenAI and Anthropic alone will deploy over 100 gigawatts of compute by 2030.
“I think by 2030, just OpenAI and Anthropic will have over 100 gigawatts combined, and then you'll add Meta and Google and so on and so forth. A humongous amount of compute”
OpenAI and Anthropic may eventually consume half of global incremental compute production.
“Hey, when does half of the world’s incremental new compute go to just OpenAI and Anthropic?”