Patel says memory now represents 30-50% of GPU manufacturing cost and NVIDIA applies 4x markup.
“And and now today, if you look at it, the cost of memory is 30 to 50% of the GPU cost. Great cost. And then obviously, NVIDIA does a four x markup on their”
Patel says NVIDIA is shipping more GPU flops this year than in its entire data center history combined.
“There's more GPU flops shipping this year that NVIDIA shipped their entire history for the data center.”
Patel explains NVIDIA strategically allocates GPUs to new entrants like Inflection to maintain multiple customers and competition.
“it was like, oh, well, who's this random startup? Like, do I wanna get them 22,000 GPUs this year? Well, actually, yeah.”
Patel states NVIDIA will build and sell approximately 400,000 H100s in Q3.
“they're gonna build at about 400,000 GPUs and and sell about 400,000 h one hundreds in q three.”
Patel forecasts NVIDIA will ship over one million H100 and A100 GPUs this year.
“NVIDIA is gonna ship over a million, h one hundreds plus a a one hundreds this year.”
Patel says NVIDIA will sell over 3 million total GPUs next year and over a million H100s this year.
“NVIDIA is gonna sell well over 3,000,000 total GPUs next year, over a million H100s this year alone. There's a lot of GPU capacity coming online. It's an incredible amount.”
Patel predicts NVIDIA will announce a new chip in March that is three to four times better than current generation.
“NVIDIA is releasing a new chip, you know, in you know, they're gonna announce it in March and they're gonna release it, you know, and ship it, you know, q two, q three next year anyways. Right? And that chip will probably be three or four times as good.”
Patel states NVIDIA will sell over one million H100s this year and over three million total GPUs next year.
“NVIDIA is gonna sell well over 3,000,000 total GPUs next year, over a million H100s this year alone. There's a lot of GPU capacity coming online.”
Patel says China still receives over a million NVIDIA H20 GPUs annually despite October sanctions.
“Right? Because you have you have sanctions on how many NVIDIA GPUs you can get in. Now, they're still north of a million a year.”
Patel says China could centralize over a million NVIDIA H20 chips into one data center if scale-pilled.
“they can centralize the chips like crazy. Right now, oh, oh, million chips that NVIDIA shipping in q three and q four, the h twenty, let's just put them all in this one data center.”
Patel claims four to five percent of NVIDIA GPUs fail during initial burn-in testing within two weeks.
“Buy a good chip from NVIDIA, and then it ends up failing like constantly, right?”
Patel says xAI's Memphis facility is the first large-scale water-cooled NVIDIA GPU deployment.
“this is the first deployment of NVIDIA GPUs that is water cooled. Right? Everyone else has done primarily NVIDIA GPUs that are air cooled, and next year, they'll do water cooled NVIDIA GPUs.”
Patel says Google deployed water cooling years before NVIDIA and achieves higher reliability than NVIDIA GPUs.
“Google's brought in water cooling for years. Right? NVIDIA only just realized they needed water cooling on this generation. And Google's brought in a level of reliability that NVIDIA GPUs don't have.”
Patel estimates Blackwell delivers 10-15x cost improvement for inference despite NVIDIA claiming 30x at GTC.
“But now, like, Blackwell, NVIDIA's pitching 10 to 15 x improvement in cost. It's like, well, you know, they're massaging the numbers marketing.”
Patel reveals each country is capped at 50,000 GPUs for four years while NVIDIA makes 6 million annually.
“There is there is one obvious loophole, which is well, there's, like, strict caps. Right? Like, each country can only buy 50,000 GPUs for the next four years.”
Patel says NVIDIA made 4 million GPUs last year and will produce 7 million this year.
“Nvidia made over 4,000,000 GPUs last year, they're making over 7,000,000 this year, right? High end data center GPUs.”
Patel explains AlexNet was the first recognizable AI model, trained on NVIDIA GPUs because CPUs were too slow.
“This is because, well, CPUs were really slow because they're really single focused on a single task. GPUs are focused on doing many tasks at once.”
Patel explains Bumpgate issue where thermal expansion differences caused GPU solder ball failures in Apple laptops.
“And what ended up happening is because of that different rate of expansion, the solder balls connecting the chip and the board would crack.”
Patel describes the Bumpgate incident where NVIDIA laptop GPUs had defective solder balls.
“There's a generation of NVIDIA GPUs for laptops. Right? And and chips have solder balls on the bottom. Right?”
Patel explains AI hardware companies made architectural bets that failed when model architectures evolved in unexpected directions.
“But still the model's way too big to fit on it. This is, like, very simple. Right? You know, the same thing's happening in the other direction.”
Patel reveals at least two AI hardware companies made wrong bets as models shifted to sparse MOEs.
“This is actually the case with at least two of the AI hardware companies today. Don't shittock them just because, you know, it's a let's be friendly.”
“But, like, you know, can you give them no GPUs? No. They're gonna retaliate. Like, there is a middle ground, and, like, Huawei is eventually going to have a lot of production capacity,”
Patel argues competing with NVIDIA requires leaping forward beyond supply chain advantages across networking, memory, manufacturing and components.
“They're gonna have better negotiations with whether it's TSMC or SK Hynix and the memory and silicon side or all the rack people or, like, copper cables, everything, they're gonna have better cost efficiency.”
Patel suggests NVIDIA should use its massive tax bill to invest directly in data center infrastructure despite customer conflicts.
“Now this is obviously gonna be, like, crazy because, like, now they're buying g p their own GPUs and putting them in data centers and doing stuff, and they're competing with their own customers,”
Patel forecasts NVIDIA revenue exceeding $300 billion next year as infrastructure spending reaches nation-state scale.
“NVIDIA's revenue this year is gonna be, like, over $200,000,000,000, and next year expects over 300,000,000,000 plus Google's gonna spend, like, $50,000,000,000 on TPU data centers.”
Patel lists NVIDIA's structural advantages across networking, HBM, process nodes, speed to market, and supplier negotiations.
“NVIDIA's gonna have better networking than you. They're gonna have better, HBM. They're gonna have better process node. They're gonna come to market faster.”
Patel explains competitors need 5x hardware advantage over NVIDIA but risk failure if AI workloads shift before shipping.
“So now you need to do something, you know, that will give you five x advantage, right, in hardware efficiency for a certain type of workload, and then pray the workload doesn't shift.”
Patel notes new NVIDIA servers require 140 kilowatts per rack versus prior CPU data centers at kilowatts.
“Data centers for CPUs, 10 megawatts of rack, 12 megawatts of rack. Kilowatts. The new ones, kilowatts, right? Kilowatts of rack, sorry. The new NVIDIA servers, 140 kilowatts of rack.”
Patel states GB 200 servers from Nvidia cost well over $3 million, preventing time-slicing users on individual servers.
“These servers cost hundreds of thousands of dollars if nothing new. Know GB 200 from Nvidia cost well over $3,000,000 so the cost of these things is huge”
Patel says GB 200 servers from Nvidia cost well over $3 million, preventing individual user time slicing.
“Know GB 200 from Nvidia cost well over $3,000,000 so the cost of these things is huge and individual users can't splice these servers.”
Patel estimates NVIDIA had north of $20 billion in China H20 revenue that had to be written off after ban.
“Our our revenue estimate for NVIDIA in China for just h 20 was north of 20,000,000,000 because that's what they were booking in capacity slash had to write off.”
Patel recounts that NVIDIA ordered Xbox production volume before receiving Microsoft's official order.
“No. No. No. Like, NVIDIA ordered the volume for the Xbox before Microsoft gave them the order.”
Patel shares industry story that Jensen ordered Xbox production volume before winning Microsoft's official contract.
“NVIDIA ordered the volume for the Xbox before Microsoft gave them the order. They're just like they're he was just like, fuck it, Yolo. Yeah.”
Patel estimates NVIDIA had over $20 billion in H20 China revenue booked before ban and write-off.
“Our our revenue estimate for NVIDIA in China for just h 20 was north of 20,000,000,000 because that's what they were booking in capacity slash had to write off. Yeah. And then it got banned.”
Patel says NVIDIA typically gets chip designs right on first silicon revision unlike competitors requiring multiple revisions.
“And a lot of times, even if the they'll start production of the you know, the a is basically the transistor layer, then the numbers, like, the wiring that connects all the transistors together.”
Patel reveals NVIDIA added tensor cores to Volta chip just months before fabrication after seeing AI demand.
“And they added the tensor cores to Volta, like, only a handful of months before they sent it to the fab.”
Patel reveals NVIDIA's next generation will split inference into separate context processing and decode workloads, not training versus inference.
“NVIDIA's next generation actually has something very different. They're not saying, Hey, there's a training GPU and an inference GPU, right? Because either is fine.”
Patel explains Blackwell introduces third memory tier within Tensor Cores requiring complex programming model for full performance.
“Now with Blackwell, there's actually even a third tier of memory, which is memory within the Tensor Core.”
Patel reveals NVIDIA doubled system-level testing time for Blackwell compared to Hopper generation.
“NVIDIA learning from the issues on Hopper, with Blackwell, they actually doubled the time for system level test.”
Patel explains NVIDIA is splitting inference into decode and prefill workloads, but provisioning for unknown future ratios is challenging.
“what sort of NVIDIA's pitching is like a split of inference into two workloads, and we'll see if they're successful. There's a lot of challenges on the infrastructure side.”
Patel reports AMD MI355 beats NVIDIA B200 in some document processing scenarios with open source software.
“in some cases, AMD actually does have a publicly usable open source implementation that beats NVIDIA even, right, with the MI355 versus V200, which is a surprise, right?”
Patel lists six current GPU types benchmarked with NVIDIA B300/GB300, Google TPU and Amazon Trainium coming soon.
“And soon, NVIDIA will be contributing B300, GB300 to us so that we can run this benchmark nightly, as well as some other clouds.”
Patel reports AMD performance improved dramatically over two months, similar to NVIDIA Blackwell's software optimization trajectory.
“Over two months, the performance of AMD dramatically improved because they went from, hey, software problems are always a thing, to over a two month period, they've dramatically improved.”
Patel states NVIDIA takes 5x markup on manufacturing cost, making power differences more significant for AMD deployments.
“If it's another deployment, if it's AMD, then that that that 20% power difference might translate to eight or 9% TCO difference when you when you talk about power cost and data center capacity cost.”
Patel reports AMD MI355 beats NVIDIA B200 on performance-TCO with publicly usable open source software in certain use cases.
“AMD actually does have a publicly usable open source implementation that beats NVIDIA even, right, with the MI355 versus V200, which is a surprise, right?”
Patel quantifies 20% power efficiency translates to only 4% TCO difference when NVIDIA takes 5x manufacturing markup.
“if you have enough power, a 20% difference in performance per watt only ends up being a 4% difference in TCO. So, in most cases, don't on an NVIDIA deployment, right?”
Patel claims NVIDIA takes a 5x markup on manufacturing cost, making power efficiency less significant for TCO.
“So, in most cases, don't on an NVIDIA deployment, right? That's where NVIDIA takes 5x markup on their manufacturing cost, right?”
Patel quantifies that 20% performance per watt advantage translates to only 4% TCO difference on NVIDIA deployments.
“But capital is far more of a limiting factor. Because if you have enough power, a 20% difference in performance per watt only ends up being a 4% difference in TCO.”
Patel reports 10-15% of NVIDIA GPUs fail and require RMA within first two weeks of cluster deployment.
“When you first turn on the cluster, about ten to fifteen percent of them fail RMA in the first two weeks. Wow. And then that's fine. Like you have to receipt them, whatever.”
Patel argues NVIDIA's divergent product lines reveal uncertainty about AI's direction.
“But really it screams, oh crap, we don't really know exactly where AI is going, which I don't think anyone does. Right?”
Patel says 10-15% of NVIDIA GPUs fail and need RMA in the first two weeks after cluster deployment.
“When you first turn on the cluster, about ten to fifteen percent of them fail RMA in the first two weeks. Wow. And then that's fine.”
Patel says Hopper GPU failure rates improved to 5% while Blackwell remains at 10-15%, expecting higher rates for next generation.
“hopper's now at 5%, but black belt's still 10 to 15%. Wow. Right? Actually started out higher than that. Sure. And when a new generation comes out, it's gonna be higher than 15%.”
Patel says NVIDIA must be 2x better than competitors to justify their 75% plus margins.
“NVIDIA recognizes they're they're the leader, they're the tent pole. Hey, in one respect, they can just run faster than everyone, but it's kind of hard to be two x better than Google or or OpenAI or whoever else's internal chip, right, to justify their, you know, 75% plus margins.”
Patel explains NVIDIA acquired Grok to get engineering resources for multiple chip architectures.
“acquiring Rock is like how you get those resources to make more solutions for different parts of the market. And as far as like, are they threatened?”
Patel explains Jensen fears specialized chips could undercut NVIDIA's margins if they only made general-purpose GPUs.
“Jensen is very paranoid about losing. Right? These specializations if he just kept making his mainline chip would mean people could you know point point solutions for specific parts of the market would crush him on cost and performance, then he can't justify his margin.”
Patel says NVIDIA must be 2-4x better than competitors to justify 75% margins and 4x pricing above costs.
“And then they have to be two x to four x better to justify four x better to justify their margins because that's what they're charging above cogs.”
Patel reports 15% of deployed Blackwell GPUs require RMA within first deployment period.
“Even today, 15% of Blackwells or so that get deployed have to be RMA'd. You have to take them out.”
Patel argues NVIDIA is setting up supply chain to manufacture tens of millions of AI chips.
“I can't buy millions, tens of millions, which is what Jensen's, setting his supply chain up for.”
Patel says NVIDIA has locked up over 60% of supply chain capacity this year in long-term contracts.
“The NVIDIA has locked up over 60% of the capacity this year in long term contracts alone, and they're buying more on top of that.”
Patel reports NVIDIA is negotiating over $250 billion in supply contracts across components this year.
“And as you look at what they're negotiating in the market today, there's over 250,000,000,000 of wafers, of memory, of substrates, of PCBs, of networking equipment that they're going to sign this year,”
Patel contrasts competitors' limited scale with NVIDIA's supply chain designed for tens of millions of chips.
“I can buy 10,000. I could buy a 100,000. I can't buy millions, tens of millions, which is what Jensen's, setting his supply chain up for.”
Patel says NVIDIA has locked up over 60% of market capacity in long-term contracts this year.
“NVIDIA has locked up over 60% of the capacity this year in long term contracts alone, and they're buying more on top of that.”
Patel reports NVIDIA will sign over $250 billion in supply contracts this year across components.
“as you look at what they're negotiating in the market today, there's over 250,000,000,000 of wafers, of memory, of substrates, of PCBs, of networking equipment that they're going to sign this year,”
Patel reveals NVIDIA acquired Grok partly because Grok chips are made on Samsung, accessing non-TSMC capacity.
“Part of it was because they want to have really fast inference, but like part of it is that Grok is manufactured on Samsung. Right?”
Patel says AI chips consuming all TSMC 3nm and 2nm capacity is forcing companies to alternative fabs.
“More importantly is that AI is buying all the capacity on three nanometer and two nanometer in a couple years that, people are having to turn to other directions.”
Patel argues NVIDIA acquired Grok partly because it's manufactured on Samsung, avoiding TSMC capacity constraints.
“Part of it was because they want to have really fast inference, but like part of it is that Grok is manufactured on Samsung.”
Patel says AI chips are consuming all three-nanometer capacity, forcing others to alternative fabs.
“AI is buying all the capacity on three nanometer and two nanometer in a couple years that, people are having to turn to other directions.”
Patel explains NVIDIA acquired Grok partly because it manufactures on Samsung due to no TSMC 3nm capacity available.
“Because there's no three nanometer capacity for them at TSMC, they need to chip somewhere else, and if you just if AI is as crazy as like we believe it is, and demand is as crazy as we believe it is, it's gonna be even crazier next year,”
Patel says Nvidia and AMD both lie about peak flops specs which are impossible to achieve.
“All their quoted specs are lies impossible to achieve Whether it's Nvidia or AMD, neither of them you can ever hit their peak flops.”
Patel reports AMD and Nvidia engineers treat Inference X as a competitive leaderboard.
“AMD and Nvidia. They love to compete with each other. And AMD engineers and Nvidia engineers look at Inference X as a leader board.”
Patel explains NVLink connects only 72 GPUs while Google ICI connects 8,000 chips without switches, creating architectural trade-offs.
“NVIDIA, the NVLink can only connect 72 GPUs. For Google, their ICI can connect 8,000 chips at super high bandwidth, but you have to pass through other chips to get there because there's no switch.”
Patel contrasts Nvidia's NVLink connecting 72 GPUs via switches versus Google's ICI connecting 8,000 chips without switches.
“For Google, their ICI can connect 8,000 chips at super high bandwidth, but you have to pass through other chips to get there because there's no switch.”
Patel says SemiAnalysis initially doubted Jensen's 25x Blackwell claim, predicting only 15-20x improvement.
“Jensen, when he originally launched Blackwell, had claimed it would be a 25x improvement. And at the time, no one believed him. Right? It's Jensen, right?”
Patel emailed Jensen documenting skepticism about Blackwell's 25x claim from industry observers.
“I emailed him. I like, hey, Jensen. You know, back in 2024, you said or back back when you launched Blackwell, said '24 twenty five x.”
Patel predicts Nvidia Vera Rubin will have smoother deployment than Blackwell due to GB's new architecture problems.
“It's it seems like it'll be a much smoother ramp than GB. You know, GB had a lot of problems because it was brand new,”
Patel argues all successful non-NVIDIA AI hardware adoption has come from labs like Anthropic and OpenAI, not enterprises.
“all of the successful AI hardware adoption of non NVIDIA flavors has been Anthropic adopting TPUs. Obviously, Google doing their own with their TPUs. Or Anthropic and OpenAI now adopting Trainium, or OpenAI adopting Cerebras.”
Patel says SemiAnalysis reported CPO on accelerator scale-up, not scale-out switches
“@pequityresearch We said CPO on accelerator scale up. We never said CPO on scale out switches.”
Patel says Rubin report was written by him and Myron, not a junior's expectation
“@kospiev @phithetasigma False. It wasnt an expectation. It was me and Myron, the 3rd person at SemiAnalysis who wrote this. You obviously didnt read any the notes re timing either”
Patel says OpenAI's Jalapeño beats NVIDIA Blackwell and Rubin, unusual for first-generation chips
“OpenAI Jalapeno is spicy Usually first generation chips aren't competitive, but OpenAI is beating Nvidia Blackwell and even Rubin This is huge news! We got to go into OpenAI's lab to dissect their new chip and dive into the software, architecture, and performance Incredible work! https://t.co/Q7Okgbxnp0”
Patel says OpenAI's Jalapeño beats NVIDIA Blackwell and Rubin, unusual for first-generation chips
“OpenAI Jalapeno is spicy Usually first generation chips aren't competitive, but OpenAI is beating Nvidia Blackwell and even Rubin This is huge news! We got to go into OpenAI's lab to dissect their new chip and dive into the software, architecture, and performance Incredible work! https://t.co/Q7Okgbxnp0”
Patel defends SemiAnalysis against bearish NVIDIA accusations by pointing to their unit and revenue estimates
“@Tanner_Invests @GavinSBaker We aren't. Look at our unit and revenue estimates https://t.co/iNAMUIHSWD”
Patel defends SemiAnalysis against bearish NVIDIA accusations by pointing to their unit and revenue estimates
“@Tanner_Invests @GavinSBaker We aren't. Look at our unit and revenue estimates https://t.co/iNAMUIHSWD”
Patel argues first-party ASICs have no residual value if OpenAI goes bankrupt, making them unfinanceable
“@TShirtnJeans2 @GavinSBaker @maxkan Why would someone finance a first party ASIC that has no residual value if OAI is bankrupt”