“So, you know, that that pricing has literally four x on the memory cost. And and now today, if you look at it, the cost of memory is 30 to 50% of the GPU cost.”
Patel states NVIDIA will build and sell approximately 400,000 H100s in Q3.
“they're gonna build at about 400,000 GPUs and and sell about 400,000 h one hundreds in q three.”
Patel forecasts NVIDIA will ship over one million H100 and A100 GPUs this year.
“NVIDIA is gonna ship over a million, h one hundreds plus a a one hundreds this year.”
Patel says NVIDIA will sell over 3 million total GPUs next year and over a million H100s this year.
“NVIDIA is gonna sell well over 3,000,000 total GPUs next year, over a million H100s this year alone. There's a lot of GPU capacity coming online. It's an incredible amount.”
Patel estimates 10 companies will have enough compute to beat GPT-4 within six months, requiring about 7,000 H100s.
“there's like 10 companies that have enough compute in one single data center to be able to beat GPT-four. Right? Like straight up. Like, if not today, within the next six months.”
Patel states NVIDIA will sell over one million H100s this year and over three million total GPUs next year.
“NVIDIA is gonna sell well over 3,000,000 total GPUs next year, over a million H100s this year alone. There's a lot of GPU capacity coming online.”
Patel notes CoreWeave got H100 services online six months before every hyperscaler in many cases.
“Get services online with h one hundreds before every hyperscaler by a factor of, like, as much as, like, six months in many cases.”
Patel finds 10x performance TCO advantage for Blackwell over H100 at certain operating points.
“But at other points, you can have a 10x performance TCO benefit from going with a more expensive Blackwell server than a cheaper H100 server.”
Patel says H100 cannot physically reach advertised 2,000 teraflops, maxing at 1,300 teraflops for matrix operations only.
“In fact, it's physically impossible to get 2,000 teraflops out of the H100. Even though the advertises it, at most you can get 1,300 FP8 floating point eight teraflops.”
Patel reports seeing AI labs sign GPU rental deals at $2.40 per hour for H100s, well above initial $1.40 cost.
“I've seen deals where certain AI labs I'm be a little bit vague here for a reason, have signed at as high as $2.40 for two to three years.”
Patel reports H100 rental prices increased from $1.71 per hour to over $2.40 in six months.
“And just in the last six months, it's gone from, you know, deals for one year transacting at $171.60 an hour for h 1 hundreds to now $2.40 plus.”