Patel reports Meta's 16,000 GPU cluster had failures every few hours while next-generation models train on 100,000 GPUs or more.
“And that was a measly 16,000 GPUs. Right? And when you look at like the next generation Llamas, the next generation GPTs, etcetera, they're training on a 100,000 GPUs or more.”
Patel states OpenAI has over 500,000 GPUs for R&D but GPT-5 pre-training uses under 100,000 GPUs.
“OpenAI has over 500,000 GPUs working on r and d. Right? Let's call it r and d.”