Patel says memory now represents 30-50% of GPU manufacturing cost and NVIDIA applies 4x markup.
“And and now today, if you look at it, the cost of memory is 30 to 50% of the GPU cost. Great cost. And then obviously, NVIDIA does a four x markup on their”
Patel argues doubling lithography costs only increases total chip cost by 10 percent, enabling five nanometer.
“If seven nanometer was completely economical, five nanometer, if the lithography element costs twice as much with the 1980i versus the next generation lithography tool, then that on the total chip basis doesn't cost anywhere close to that, right? Because again, the manufacturing cost of lithography is only one fourth to one fifth.”
Patel calculates doubling lithography cost only increases total chip cost 10 percent, enabling five nanometer.
“Because again, the manufacturing cost of lithography is only one fourth to one fifth. So if you double lithography costs, you're only increasing the total cost of a chip by 10%, right?”
Patel argues training costs are irrelevant, noting GPT-4 used 20,000 A100s.
“training costs are irrelevant. Right? Like, GPT four, right, like, 20,000 a one hundreds, that's that's like, I know it sounds like a lot of money.”
Patel argues AI demand is what makes continued semiconductor node advancement economically viable today.
“funding the next node would not be economically viable anymore if it weren't for AI taking off. Right? And then generating all this humongous demand for the most leading edge chip.”
Patel states OpenAI is paying $12.5 billion over five years for one-fifth of Oracle's data center site.
“So for one fifth of the site, they're paying something like $12,500,000,000 over the next five years.”
Patel breaks down full H100 GPU cost at $40,000 to $45,000 including networking and infrastructure versus $24,000 for chip alone.
“each GPU, right, once you include networking, building, all this sort of stuff maybe is you know, the GPU itself of h 100 is, like, 24,000, but once you add everything else up, it's, like, forty, forty five thousand dollars per GPU all in of everything.”
Patel argues that efficiency gains alone without capability increases cannot justify massive AI infrastructure investments.
“That would not pay for all of these build outs. Right? AI is useful today, but it's not capable of doing a lot of things.”
Patel describes Chinese manufacturers as locked in constant cost reduction cycle rather than margin expansion.
“there's this constant pressure cooker of continual cost reductions and engineering. It's not like, Oh, well, we make 80% margins. Our cost is low.”
Patel states 80% of GPU data center cost is capital equipment, only 20% is land, power, and cooling.
“It's the it's the physical data center conversion power conversion equipment. All of this stuff is, like, 80% of the cost. And then 20% is gonna be your land and your power and your cooling”
Patel says as the largest technology Substack, the 10% platform fee is worth paying for the growth.
“Even though a 10% cut sounds like a lot, I'm the largest technology substack and I'm telling you that it's worth it.”
Patel states AI infrastructure represents 25% to 75% of current US quarterly economic growth.
“It's anywhere from 25% to 75 of the current quarter's economic growth. That's insane for The US.”
Patel says avoiding repeated pre-fill on 30,000-token inputs can reduce inference costs by 75% through KV cache.
“So then that 30,000 tokens, that pre fill, that 30,000 tokens still cost a lot more than that output.”
Patel explains users will pay 10x more for 10x faster inference completion, justifying Cerberus economics.
“for a lot of people, I'm fine to spend 10x the price on something that completes 10x faster. So Cerberus sort of just makes a ton of sense there.”
Patel notes 2% of GitHub commits are AI-generated against $2 trillion in global software wages.
“You can disable that where it's not automatically committed. But 2% of GitHub commits today are Cloud Code. $2,000,000,000,000 of software wages paid in the world.”
Patel calculates $100B AI revenue requires $250B infrastructure spend at five-year depreciation, double current capex.
“That $50,000,000,000 of COGS needs to burn on infra, which cost roughly with if a five if you're talking about five year depreciation, call it $250,000,000,000, right, of infra Yeah. For a $100,000,000,000 of revenue.”
Patel observes perception shift where 90th percentile people consider themselves middle class while 50th percentile people do not.
“Nowadays, people in the ninetieth percentile think they're middle class, and people in the fiftieth percentile don't think they're middle class.”
Patel says he now supports UBI despite being very capitalist, a shift from his prior views.
“Over the last couple years I've realized, wait, I actually think UBI is perfectly fine. Which I think is crazy because again, I'm very capitalist.”
Patel cites estimates of 20% economic decline if Taiwan is invaded, comparable only to world wars.
“And then subsequently, that that sends the world into a depression. Right? The world has not experienced a 20 free fall since maybe, like, a one of the great wars.”
Patel states GPU rental rates were $12-13 billion per gigawatt before recent increases.
“GPUs, at least before the craziness of the last six months, usually went around 12 to $13,000,000,000 per gigawatt.”
Patel explains KV cache storage and reuse enables massive cost decreases in inference.
“you calculate that once, you store it off in memory, whether it be system memory or storage, and then you pull it back in when you run the turn.”
Patel says OpenAI's total gross margin rose from 30% to 55% over the past year.
“OpenAI late last year, their margins had were roughly 30% gross margin, but if you stripped away the free users, they were at 50%. Now, total company gross margin is closer to 55%,”
Patel reports that tokens now represent 30% of his 90-person company's costs versus 70% for employees.
“My my own company of 90 people, 30% of my cost now is tokens versus 70% employee costs.”
Patel argues hyperscalers would pay 8% interest rates versus current 5-6% given AI compute returns.
“Meta's raised at, like, 5% to 6%. I don't see why they wouldn't pay 8% Because they would happily pay 8% because the return from the compute that they're going to build is humongous.”
Patel reports Anthropic now generates up to $50 million per megawatt, a 5x return on compute costs.
“And and what that now enables them to do is, hey, if I spend $10 on inference capacity, actually generate $50 of revenue,”