FROM THE RECORD · 3 matching entries · 2025-03-25 to 2026-08-25
What does the record say about interest rates?
Selected points from existing editorial summaries and source excerpts. This covers the material on this page.
1
Patel says commercial real estate investors now finance GPU clusters at 12% interest rates like standard real estate. [1]
Read the supporting quotation · Prime Intellect AI · 2025-03-25
“Like literally real estate investors, commercial real estate investors are doing notes for GPU clusters at like 12% interest rates which is just like it's like standard commercial real estate now.”
Patel argues Meta could pay 8% interest rates versus current 5-6% because compute returns are enormous. [2]
Read the supporting quotation · Dwarkesh Patel · 2026-08-25
“So why wouldn't interest rates Yep. For Amazon go from, you know, from where they are today? I think Meta pay okay, let's like so this is going be extremely lived out.”
FROM THE RECORD · 76 matching entries · 2023-02-02 to 2026-07-22
What does the record say about nvidia?
Selected points from existing editorial summaries and source excerpts. This covers the material on this page.
1
Patel reveals each country is capped at 50,000 GPUs for four years while NVIDIA makes 6 million annually. [1]
Read the supporting quotation · Unsupervised Learning: With Jacob Effron · 2025-01-21
“There is there is one obvious loophole, which is well, there's, like, strict caps. Right? Like, each country can only buy 50,000 GPUs for the next four years.”
Patel explains NVIDIA acquired Grok partly because it manufactures on Samsung due to no TSMC 3nm capacity available. [3]
Read the supporting quotation · Daytona · 2026-04-07
“Because there's no three nanometer capacity for them at TSMC, they need to chip somewhere else, and if you just if AI is as crazy as like we believe it is, and demand is as crazy as we believe it is, it's gonna be even crazier next year,”
Patel predicts OpenAI will raise 50 to 100 billion dollars by early next year to fund planned cluster builds. [3]
Read the supporting quotation · Dwarkesh Patel · 2024-10-02
“there's no fucking way you can pay for the scale of clusters that are being planned to be built next year for OpenAI until unless they raise, like, 50 to $100,000,000,000, which I think they will raise that, like, end of this year, early next year.”
Patel reveals Anthropic turned first profit in Q2 and will hit billion-dollar operating profit in Q3 before IPO. [2]
Read the supporting quotation · RAISE Summit · 2026-07-16
“So Anthropic turned their first gross profit in Q2 in June. And then in Q3, they will be turning a billion dollars of operating profit, slightly over.”
Patel's firm's AI spending jumped from under $100k to $11 million annually within months. [3]
Read the supporting quotation · WisdomTree in Europe · 2026-07-09
“by the end of January, our ARS, right, our our our annual recurring spend had hit $4,000,000. And that's because people were using Cloud Code. Now today, it's about $11,000,000.”
FROM THE RECORD · 40 matching entries · 2025-08-14 to 2026-08-25
What does the record say about anthropic?
Selected points from existing editorial summaries and source excerpts. This covers the material on this page.
1
Patel reveals Anthropic turned first profit in Q2 and will hit billion-dollar operating profit in Q3 before IPO. [1]
Read the supporting quotation · RAISE Summit · 2026-07-16
“So Anthropic turned their first gross profit in Q2 in June. And then in Q3, they will be turning a billion dollars of operating profit, slightly over.”
Patel reports Anthropic is Q2 profitable excluding SBC, expects Q3 profitability including SBC with 80% margins on Opus tokens. [2]
Read the supporting quotation · Sequoia Capital · 2026-06-30
“And I think by Q3 they may even be profitable, including stock based compensation. That's how profitable they're getting, and their margins on an Opus token, at least Opus 4.8 token, is north of 80”
FROM THE RECORD · 38 matching entries · 2023-12-05 to 2026-08-25
What does the record say about infrastructure?
Selected points from existing editorial summaries and source excerpts. This covers the material on this page.
1
Patel estimates $3-4 trillion total CapEx needed by 2028 across compute, data centers, and energy infrastructure. [1]
Read the supporting quotation · Dwarkesh Patel · 2026-08-25
“So to enable, let's say, that 100 gigawatts by 2030 or let's even like let's even like pare it down to 2028 where it's like 3 or $4,000,000,000,000 of CapEx across all of these items.”
Patel identifies a 40 gigawatt power deficit in US even with all planned gas and restarted coal. [2]
Read the supporting quotation · Special Competitive Studies Project · 2025-03-13
“Even if we don't turn off any more coal, in fact we restart some, we take all the natural gas dual combined cycle reactor deliveries that we're going to have,”
Patel predicts OpenAI will raise 50 to 100 billion dollars by early next year to fund planned cluster builds. [3]
Read the supporting quotation · Dwarkesh Patel · 2024-10-02
“there's no fucking way you can pay for the scale of clusters that are being planned to be built next year for OpenAI until unless they raise, like, 50 to $100,000,000,000, which I think they will raise that, like, end of this year, early next year.”
FROM THE RECORD · 36 matching entries · 2023-08-09 to 2026-07-09
What does the record say about supply chain?
Selected points from existing editorial summaries and source excerpts. This covers the material on this page.
1
Patel explains NVIDIA acquired Grok partly because it manufactures on Samsung due to no TSMC 3nm capacity available. [1]
Read the supporting quotation · Daytona · 2026-04-07
“Because there's no three nanometer capacity for them at TSMC, they need to chip somewhere else, and if you just if AI is as crazy as like we believe it is, and demand is as crazy as we believe it is, it's gonna be even crazier next year,”
Patel reports NVIDIA will sign over $250 billion in supply contracts this year across components. [2]
Read the supporting quotation · CNBC Television · 2026-03-16
“as you look at what they're negotiating in the market today, there's over 250,000,000,000 of wafers, of memory, of substrates, of PCBs, of networking equipment that they're going to sign this year,”
Patel predicted memory prices would soar because capacity grows 30% annually while demand doubles. [3]
Read the supporting quotation · WisdomTree in Europe · 2026-07-09
“Memory capacity is only growing 30% a year for the next three years. And yet demand is doubling, is doubling. And so what's gonna end up happening is memory prices are gonna keep soaring.”
Search covers the displayed transcript and written excerpts, plus all X posts stored for this person, including replies. Coverage is growing; this is not a complete account history.
Explore
In their own words
1394 entries
Transcript excerpts may contain transcription errors. Open the recording for wording, speaker and context. Source links do not verify the claims made.
No matching entries. Try fewer words or clear the filters.
Choose an end date on or after the start date.
X replyX
Replying to rahul
about to send this. what do you guys think https://t.co/otLloZJkju
Today I’m joining @Natl_Compute
I’ve spent the last four years working on GPU access, first as a founding engineer at Brev and then at NVIDIA after they acquired us in 2024. When I started, on-demand GPU capacity at the hyperscalers was effectively gone. If you were an https://t.co/XnMoFyEOne
Well ok then, I'll bite.
@SemiAnalysis_ is once again full of crap.
"And here’s the truth for you: onepicture, >24GW of datacenters in advanced stage of construction, all to be delivered in 2027."
No your "platform" does not have accurate tracking, nor does it have a right https://t.co/VqskdduXHT https://t.co/vcVzOefHcn
@FransBakker9812 @SemiAnalysis_ It's 195mw that goes up in 2027 as per our model, the Crusoe figures are a bit higher but we base it on what will actually be built based on our data.
This is capacity that's being built currently and some goes live in 2027 and we account for partial construction of sites.
Semianalysis saying if you’re worried about demand (ARR,) don’t be, because there’s going to be even more supply than you realize https://t.co/WTqkPfM9YH
I feel like I've been censoring myself way too much lately, and it's honestly been getting to me. I'll want to get into a debate or an argument with someone, but then I worry I'll say the wrong thing or offend someone and end up stirring up controversy, so I just can't bring
@DamianBullish @citrini Nah that's still citrini
But I defend @jukan05 too.
I don't like he's being discouraged from his opinion by clueless loud voices
I'm doing a new startup with @asciimike
It's Chinese leg extension surgery
Except for you break your skull and expand your brain https://t.co/DV6GVARxTj
i literally cannot believe that i enthusiastically use an atlassian product every single day. 600m for the only team taking prosumer browsers seriously was a bargain https://t.co/SFiybBOPI3
Curious to see how many subs @SemiAnalysis_ has in South Korea and whether they are more or less respected there than KIS, whose spec sales note was apparently a contributing factor to last nights sell off.
A modern day Clash of the Titans.
Should know in a few hours! https://t.co/mAvNP8Wuj6
@GavinSBaker @SemiAnalysis_ We were wrong so we hired her and a few others. We have 5 people in Korea. Will take down trendforce and other research houses there https://t.co/hU9k3nHIxW
The Korean analyst with the best track record of forecasting SK hynix’s earnings released a report today predicting a substantial beat versus consensus.
The analyst forecasts operating profit of KRW 80.1 trillion, above the current consensus of KRW 74 trillion—with recent
@dylan522p @obsidiancap1 You mean year end ARR of about $600B for Anthropic + OpenAi and not FY27 revenue of $600B?
Then that’s $50B in Dec 27 revenue. Doable.
@obsidiancap1 I don't care what consensus or expectations should be for IPO.
My job is to estimate what actually happens.
Also its 4.3x to 5x revenue growth, not 7x over the next 14 months.
@dylan522p @Samuel947117225 @jiahanjimliu Fine but why do you keep on hating on all of the advancements. I'm an idiot but I can see it. And I'm in your lazr bc I like the lite and the anthropic. But stop hating. Alot of good people like these stocks
.. I've watched alot of your interviews. Play it reasonable.
@joe59017 @Samuel947117225 @jiahanjimliu It’s not hating. We are grading them and proving feedback. Should the market have 0 technical discernible feedback? That sounds terrible
@dylan522p @joe59017 @jiahanjimliu they are a shareholder of Fluidstack, IREN is apparently getting rid of FS as middleman. Is this true? Are you a shareholder of Fluidstack?
@Samuel947117225 @joe59017 @jiahanjimliu Yes I am an investor in the Series A from Q1.
And we downgraded them to unavailable.
Worse than IREN rating.
Our rankings are unviased
Fluidstack never used any relevant amount of Iren capacity and has nothing to do with this downgrade. Read the report
@dylan522p @jiahanjimliu I'm 60 percent believing you. I kind of trust you. But also the optics thing.....and a few others. Will you say you arent profiting off your stated opinions.
I may have trust issues.
@joe59017 @jiahanjimliu We make money via selling data, see datacenter model, TCO model etc as examples.
We don't charge for ClusterMAX, or ratings, or testing.
It's something we do for public good and aura and proof we know the industry way better than others.
Amazon is a bigger customer than any these
$IREN: Addressing SemiAnalysis Reply
Truths
I appreciate @dylan522p for now telling the truth: "There's not a single cloud that doesn't have downtime, the question is how much, how often, etc."
This is exactly what I'm saying in my post: IREN has had downtime but other clouds https://t.co/eZTxNa3bYp
@jiahanjimliu We never blamed iren for together downgrade.
Together has no feedback shared on other clouds.
Together downgrade is because their own capacity.
Can you just read the full clustermax report.
We talk about reliability issues at many of the clouds tested.
$IREN: Addressing the Semi-Analysis FUD on Reliability
What's the Issue?
I heard of these issue in August and the only customers that $IREN has had long enough to report "multi-day power outages, network upgrades, storage failures, air quality controls leading to link flaps, https://t.co/cFHB1bIdEq
@jiahanjimliu This is actually funny as fuck
There's multiple sites with item that have had issues, not 1
TogetherAI was downgraded in ClusterMAX because of reliability issues.
Vipul is not even guju and I know him and like him but our feedback and downgrade is because of quality issues.
We
@IsaacJLiu @JKeynesAlpha No 6.1 ultrafast doesn't run on cerebras at all. All the Cerebras capacity is running older models that are optimized for Cerebras
Meet Mistral Large 4, aka Le Chonk.
• 1T parameters, natively multimodal. 49B active.
It is the best open weights model from US or Europe on aggregated benchmarks.
• State-of-the-art on critical workloads, including cyber defense, manufacturing and finance and it surpasses https://t.co/Jr5Z11mxEv
Mistral 4 is pretty good at lawyer bench and cyber security bench, but give it's European heritage, one can only assume it's also good at regulation bench https://t.co/YeS7lUpb0V
We get a lot of questions about CS-4’s components.
Take a closer look at the engineering behind the CS-4 Wafer-Scale Backpack. ⤵️ https://t.co/Z246whpNMv
I really really really wish we had more frequent and reliable updates on AI spending and revenue trends. I see this as probably single best metric for measuring *the* key trend in the global economy, and we’re reading it off advertorial charts from random payment processors :/ https://t.co/pvnyIxBR8i https://t.co/ovrWmr3bjI
"And antisemitism will be dead in about 6 months once Sydney Sweeney converts so don't worry bout it" - @Chaimeisenberg
Incredible things are happening in SemiAnalysis Slack https://t.co/MiA0BsDdcX
This Semi analysis piece is not the full picture, I wonder why they didn't show the full picture, got money in the IPO Dylan?
1. The models burn through tokens differently. Artificial Analysis reports about 119,000 output tokens per Intelligence Index task for Opus 5.5 at max https://t.co/NuCXgeDyYr https://t.co/jmbzzrNfwT
It doesn't make sense to say a subscription plan is worth $ X in isolation.
The way to think about subscriptions is that your monthly payment grants you some numbers of “credits”. Each (model, token type) combo consumes a different amount of credits. Because credit cost ratios https://t.co/u8wyQH4rUR
The difference in what Anthropic and OpenAI as well as many others give you in value for a subscription is insane.
We tested Meta, Minimax, Kimi, Grok, and Cursor subscriptions https://t.co/fbZAJ3wS96
@dylan522p Dont worry plenty of other people accuse you of much more unethical things these days. Snide remarks from me are least of your problems.
And our mutal friend got $8.6K/month API cost of $200/month sub instead of your $11.7K. Likely due to more optimistic caching assumptions.
@insane_analyst Huge difference in numbers, so if it's so easy why is variation is so large.
As far as accusations, those aren't people I considered friends and help.
Making shit up about me is a huge fuck you, so fuck you right back
@dylan522p It takes a month to calculate basic info that a mutual friend of ours did out of curiosity some time ago with a single AI agent in less than a day?
@insane_analyst We had to retest many many times because methodology has to be refined constantly
If things were easy everyone would do it
You can see we've have old data that we tested a while ago
Accusing me of unethical things with 0 evidence is a giant fuck you
Not gonna help you ever again https://t.co/OOfRE6uspr
@insane_analyst It's objective analysis of OAI and Ant subscriptions that's been in the works for over a month.
Testing things objectively and sharing results makes us pumpers now?
Fuck off
Introducing Beam: a highly efficient agentic open model with 501B total parameters and 23B active.
- Frontier reasoning efficiency
- Advances the Western open frontier on coding & agentic tasks
- Trained end-to-end from scratch
Full weights release this month.
Learn more about https://t.co/1rMABCywUG
@dylan522p @RegularShep @bookwormengr @SemiAnalysis_ @sama Please explain how “what does this say about cerebras?” Is not 100% a speculative open ended question.
I made this post after seeing someone named Dylan Patel on my timeline but turns out he is famous so maybe name your kid Scott Chennapragada he could turn out successful like that guy did
@dylan522p @JKeynesAlpha Cbrs is really expensive tbh. But at the same time Semianalysis was bearish on AMD at $120 cuz cutting they don’t write software. But AMD has 5xed since.
@dylan522p @bookwormengr @SemiAnalysis_ Ok i guess @sama just posting randomly in the fucking ether. Respect, but this is petulant child nonsense. Do better. https://t.co/TD8DQ3jPla
@RegularShep @bookwormengr @SemiAnalysis_ @sama We never said Cerebras wasn’t a partner? Just that 6.1 sol ultra fast is on GPU.
We never speculated on the relationship just posted facts. Everyone else speculated
Wait what??
Nah but surely there is no way my maths are right:
OpenAI is allegedly selling Cerebras-powered Ultrafast inference to Jane Street at
*check notes*
~$200M per MW per year?!
$200,000,000 PER. MEGAWATT.
Now remember:
OpenAI is contractually set to receive 250 MW
@beniduboss That implies it's all delivered and fully utilized.
Chaining. Dozens and dozens of wafers together brings quite a few offlinenand utilization issues
@dylan522p @ilzmcfly @vuuvies Pricing differential to Chinese distributors is a different class of problem than 'chips leaked through the supply chain.' One is control failure. The other is commercial intent. AMD already disclosed an $800M charge; this would be the harder conversation.
@ilzmcfly @vuuvies Yes it wasn't clear if they intended to sell or not to them.
The treason part is selling to distributors in China for less than large volume customers in America.
AMD didn't address that part but I've got more evidence behind it
@dylan522p You are seriously going to play this game of "not knowing" that people would easily interpret this as a negative signal for $CBRS?
Couldn't you directly address the fact in the first post itself that it is a capacity story, not a replacement one?
@dylan522p @bookwormengr @SemiAnalysis_ Why paywall the correction? Markets moved on this news. Acknowledged in the correction. You feel mo ethical obligation to platform it the same way you did the original misleading post?
@dylan522p @athuinvests Why didn’t you include the full story then? That’s it’s running on Nvidia AND they’re not moving off Cerebras. That would be professional
@SemiAnalysis_ SemiAnalysis was publicly bearish on Micron’s HBM positioning when $MU was around $300, even suggesting Micron could have zero share of NVIDIA Rubin HBM. Now Micron currently trades at $1000+
That call is worth remembering when evaluating their latest claims around Cerebras. https://t.co/nzTY48lJBX
@Venu_7_ @SemiAnalysis_ We posted facts.
Micron couldn't hit the original pin speed targets for Nvidia.
Nvidia relaxed the pin speed targets and HBM bandwidth for Rubin.
They got back in.
We reported all these things sequentially but people see 1 and mins interpret as everything
When we said micron
Pretty greasy stuff
“We believe that the market falsely misunderstood this as OpenAI moving off Cerebras”
Gee whiz wonder how the market got that impression https://t.co/3AJYfkAJ4x
@negligible_cap We never said OpenAI moved off Cerebras?
We said 6.1 Sol Ultra Fast runs on GPUs
That's a fact.
That's your fault for taking that to mean they moved off Cerebras.
$CBRS SemiAnalysis causing the “misunderstanding,” then backpedaling behind the paywall lol 🤦♂️ Shameless. https://t.co/QMTABxGuWw https://t.co/T64OGcLSC3
what's up with AMD neoclouds?
AMD's strat has been giving chips to a few (instead of NVIDIA who spreads them out) cuz not everyone has the expertise to fix them as issues come up and it's easier to deal with problems that way.
atp AMD is harder to get than NVIDIA, and oft req 3 https://t.co/CJVjs909zr
@DirtyTesLa We cut our RAM in half for the Tesla AI5 chip (now 72GB of LP5) and 1/3 for AI6 (now 144GB of LP6).
This was the only way to get enough volume for Optimus production and greatly reduces cost.
As it turns out, we think this will have a negligible effect on Optimus performance,
Steptember is over
- Average 35.1k steps/day
- Record 70.2k steps in a single day
- Walked 498 miles (19 marathons)
- Lost 10lbs
- Feel (and look) better than ever
Excited to sit for the entire month of October https://t.co/U46TDfVBiy
@dylan522p Dylan - Chris told Factory (among ~10 other companies he was also advising) that he was joining Cognition full-time as CRO. In response, Matan asked Chris to consider a full-time role at Factory instead. Only after Chris declined did Matan decide to attack Chris's reputation.
In
A lot of people tell me they use agents to build xyz
Thing is, some people have an accent or say it quickly
And so I hear we use Asians to build xyz
Which is like...
Well ya that's been the global economy for the last few decades.
@dylan522p Hey Dill Dill. Remember when you confidently told the world OpenAI and Anthropic were beginning to turn a profit a few months back? Surely you didn't just make that shit up right?
I'm not sure why people keep saying that the AI boom is bigger than the railroads. It's getting close, and the decade average might end up bigger, but railroads hit 5.4% of GDP in the 1870s and 4% of GDP in the 1880s. https://t.co/ra7eUuQqeh https://t.co/cTLUyzxERI
SemiAnalysis on Intel 18A Density: A real banger!?
"Our measurements put Panther Lake’s 18A compute logic and TSMC N3E GPU logic at similar logic density. However, 18A does not lead TSMC N3P, N2 or Samsung SF2 in peak density."
First, Panther Lake CPU tile uses Intel 18A HP https://t.co/LLn4sxAQZp
@SemiAnalysis_ The quality of the floorplan/architecture analysis is disappointing, not as good as the process analysis. I miss @FritzchensFritz and @Locuza_ ...
@dylan522p @OInvests And also why have Akima in the same unreccomended category as IREN when you say you hate IREN just because of their industry sites??? Ironic much bc Akima that very same day just signed a $20 Billion with Anthropic for their sites. Go fuck yourselr you pussy
Fentanyl Grade Compute: Why a GB300 Rack Will Out-Price Blow by Weight in 2030
A GB300 NVL72 weighs roughly 1,580 kg fully populated and recent purchase orders put it at $5M per rack.
That is $3,165/kg. Strip out the 1.5 tons of busbar, manifold and coolant and the GPU packages https://t.co/0PX80vwguq
@knowrohit07 @rasdani_ Prime Intellect offers managed cluster services and are a neocloud in their own right
They were ranked underperform in a prior clusterMAX despite my investment because they were worse and have gotten better super fast.
Just a reminder that @SemiAnalysis_ and @dylan522p thought it was ok to steal an image from a creator, remove the watermark, and publish it as if it was their work, with a tiny little credit so they wouldnt get sued. Will continue to post this everytime they act like children. https://t.co/GU4xCa9TEp
@GlobalCollapse @SemiAnalysis_ We didn't remove any watermark, we credited the original image from day 1. We did watermark all images in our posts but we never stole our didn't credit
@dylan522p Hi Dylan, thanks for responding.
It’s bold to state what is posted here is “Fake news” when in fact you are acknowledging in this comment the point I was making in this post… you are not denying $NBIS is a customer nor are you denying a paid relationship.
The testing was
@OInvests Correct of the top 20 companies in the world 5 aren't customers.
Everyone buys our research and contracts us to do research for them.
We never market or change our opinion for money.
That's why people care what we say.
Not having managed clusters means you fail ClusterMAX because
$NBIS commissioned SemiAnalysis to model total cost of ownership across GPU workloads in March 2026.
The study, paid for by $NBIS concluded Nebius has the lowest TCO in all scenarios.
Six months later SemiAnalysis places $NBIS in the Platinum tier and rates $IREN as https://t.co/VYUfVvzuiS
@OInvests Fake news.
We’ve done work with dozens of neoclouds and their equity and credit holders.
We do not pay to play.
Google Microsoft Amazon Oracle are all larger customers of SemiAnalysis than Nebius and Coreweave.
How about you actually look at the technical side of it
Fentanyl Grade Compute: Why a GB300 Rack Will Out-Price Blow by Weight in 2030
A GB300 NVL72 weighs roughly 1,580 kg fully populated and recent purchase orders put it at $5M per rack.
That is $3,165/kg. Strip out the 1.5 tons of busbar, manifold and coolant and the GPU packages https://t.co/0PX80vwguq
We are approaching cyber-superintelligence.
But our systems aren't ready. We must secure our model weights and infrastructure against three new threats: sabotage, escape, and theft.
There is a way forward.
Secure Acceleration:
A Cyberdefense Strategy for Superintelligence https://t.co/FLNPLPVWvF
Everytime AI spend grows where I flirt with token budgeting, Anthropic + OpenAI save me
Astra had a spike then settled down lower
Opus 5.5 it looks like spend is lowered for current usecases
Of course people figure out new things to do
Spend goes up again
ROI keeps increasing
My feed right now:
- People reposting ClusterMAX rankings without reading the report
- People commenting on ClusterMAX without reading the report
- People criticizing the results without reading the methodology
- People criticizing the results while totally missing the point https://t.co/noXImLTggt
You can tell how intellectually honest someone is by whether or not they're willing to acknowledge that they made a mistake and really try to understand why they were wrong.
Was it cognitive bias? Was it bad information? Was it misunderstanding the situation, or a combination? https://t.co/wMuc78LZIn
@SaveTheEarth079 What I originally said is still true. Their parts are out there flying through cracks in their supply chain, and at lower prices than the US.
US supply chains for electronics suck so much
AMD found a business opportunity in dumping US military chips for 1/4 of the cost in China.
US list price is $36k for this chip, with $4-5k to US companies at volume, but it's getting quoted $1k in China to crowdfunding campaigns
AMD https://t.co/XjewvKJAsA
Spoke with AMD, they did not ship / sell this dual use chip to this firm and they're investigating the sourcing.
This violates their policies + they work with authorities to take action on violations
I am very concerned about the fact that pricing discrepancy is so large.
I have https://t.co/ZWmYx7KAvL
We took the silicon out of the silicon. Quick turn from SemiAnalysis STEEL Teardown Lab: iPhone 18 Pro Max, A20 silicon on TSMC N2. more to follow... https://t.co/RURlD49MP1
I wanted to use this photo because it's the actual package and looks way cooler.
The black area above is the DRAM but not milled yet.
Cross section from our Hitachi XTEM coming soon 🤫 https://t.co/iWyTyW4rRP https://t.co/RMwK2bljPP
US supply chains for electronics suck so much
AMD found a business opportunity in dumping US military chips for 1/4 of the cost in China.
US list price is $36k for this chip, with $4-5k to US companies at volume, but it's getting quoted $1k in China to crowdfunding campaigns
AMD https://t.co/XjewvKJAsA
@dylan522p and I had briefly sparred here on a couple of occasions. It's normal between people who have spent time/effort doing their work and hold strong convictions to engage in a honest, intellectual debate. Sometimes it's Rashomon, sometimes it's just one being right
While https://t.co/O4EZa4oRyC
@R_and_Invest We do have our own agents we've built for many ops things including invoicing and such.
Doesn't mean an org this large doesn't need someone guiding the wheels
We built high-throughput materials labs in Menlo Park to create a loop between experiments and models. The labs generate fresh data, the models learn from it, and then help us decide what to try next.
Using only 1,300 H200s, plus months of our experimental data, we mid-trained https://t.co/Ubvbdvg0Gi
I'm hiring 2 full time folks in Chicago in office.
Accounting
Operations
The amount of diff ops our 120 person company has is incredible
The amount of diff work we see
The opportunity to work with an incredibly fast moving and talented organization. If anyone knows anyone pls lmk
@negligible_cap Yes I believe they are not frontier and current Gemini 4 while having awesome pretraining, doesn't solve their reasoning and agent issues
.@Citrini Research has been sold to the semiconductor and AI research firm @SemiAnalysis_, founded by Dylan Patel, @dylan522p. Citrini Research founder, James van Geelen will stay on as in-charge of this Citrini unit, he plans to launch a new fund in future.
Citrini Research
@DiscountedTF I'm spending the week with credit bros in NYC.
I think they're smarter than equity guys now.
I used to think credit bros had more brain damage.
@dylan522p @inferact Does InferenceX count abandoned or cancelled requests, or only completed ones? Curious how much of the $/token gap survives when you include generation that never reached a user.
TPU Inference Externalization
Full Steam Ahead - InferenceX,
Up to 50% Better Performance per Dollar,
Rapid Externalization of TPU stack,
Growing Customer Base,
Ironwood, TPUv8i, Reducing CUDA Moat
https://t.co/9CFY8YDRsI
We are excited to bring the first open benchmarking of Google's TPUs to the world
Running every day, on many models + scenarios
$/token is better than B200 and B300
Huge shout-out to Google @inferact and the InferenceX team at SemiAnalysis to this effort that's taken many months https://t.co/vmtJl2ApV0
One of the most interesting blog posts we've released: details on internal research acceleration at @OpenAI. I expect these trends to continue.
We also share some details on how we've paced model development to prioritize monitoring, alignment, and security. https://t.co/mURAZxpVwO https://t.co/zOY6OxJhzQ
@polynoamial @OpenAI I'm kinda surprised AI company employees use AI about as much SemiAnalysis employees now, but earlier in the year they used significantly less.
IG y'all didn't have Claude Code then, but now ur models are better.
@MilksandMatcha If it's early stage and it's a nice deck without AI it's negative signal.
If it's a simple deck without AI that's good signal.
If it's a nice deck with AI that's a good signal.
i think there’s a small but real chance with the rate of internal acceleration inside frontier labs that the model after Astra isn’t called GPT-6.5 but instantly GPT-7
I was chatting with a candidate to lead sales in Asia region about how 85% of our customers are US based despite 40% the team being based in Asia.
Unc straight up said
"I think the reason you fail to sell in Japan, Korea, China is because you young kids don't drink enough"
🥲
@cdleary Do these optimization passes often fall into local minimas?
Does it require a lot of human in the loop to jolt them out?
Or are all the major things already human in the loop like hey do fp8 attention, hey do attention rescaling with tile look ahead, hey do key tile prefetch?
@Interestingna11 @damnang2 @andrewdfeldman Jalepenjo is overhyped. Im going after SA on that. LPX im going after Cerebras. Believe it or not its possible for both sides to be wrong. 🙂
Weekend project -- inference engineering step-by-step with GPT!
It made a really fun beginner tutorial with exercises covering:
- how to run the simplest working LLM server
- intuition for prefill and decode
- performance benchmarking on my system
Next lesson plan: writing my https://t.co/SFEJGYPOwx
@dylan522p @Cbk19881 you know that this is the definiton of efficiency - they dont need a full on gangway so they use less space for the aircrafts - ergo being place efficient
@MuskpromotesRac One the worst airports in the world to make a connecting flight through, takes 1.5hr + if u have a layover for security and line and walking between terminals etc
@GavinSBaker @dylan522p @maxkan Hey Gavin, can you explain to us why you think "Jalapeño not financeable at comparable rates to GPUs?"
Wouldn't a custom ASIC be much cheaper to make than paying the Nvidia margins?
@dylan522p @maxkan Maybe GPUs plus CS-4/5 for Ultrafast at scale. Jalapeño not financeable at comparable rates to GPUs and probably not that big of a wafer allocation next 2 years. Jalapeño still a really impressive achievement from OpenAI. I remember being sad when they hired Thomas away from https://t.co/a0jIXXahXN
Introducing S1, our new foundation model that learns from one example.
It can be taught 10-minute long tasks that it has never seen before, from one video prompt without any fine-tuning.
Watch S1 operate in real-time via in-context learning: https://t.co/wmF3Byv179
@maxkan Hmm. I might ask @dylan522p if Attention-FFN disaggregation (AFD) is compatible with data locality and not moving the KV cache. That constraint is what OpenAI focused on with Jalapeño. They absolutely did not make an “explicit bet that disagg is not the way to go.” There are https://t.co/2lVjH2M5vS
As the kids in SF say, @dylan522p is only 'directionally correct'. Since no one's done the math, I did:
Economics of becoming your own inference provider come down to 4 numbers:
1. Selling price: Kimi K3 charges $15 per 1M output tokens. But in a competitive market, are you https://t.co/WBJgZmnlG4
“As we go forward into the future, the the numbers for computer ballooning, right, we're at, you know, you know, a little bit over a trillion dollars of CapEx this year.”
“You've got Anthropic OpenAI are taking as much as 40% to 50% of compute next year. And the centralization doesn't look like it's slowing down or stopping. In fact, it looks like it's only accelerating.”
“So by the time you're in like towards the end of twenty twenty eight, if this trend continues, which I see nothing that's stopping it, You you've got them just controlling most of the usable, you know, flops in the world on their own.”
“So, this is sort of what I personally believe that the labs are going to allocate less and less compute to inference over time, I think is very non consensus, right?”
“So to enable, let's say, that 100 gigawatts by 2030 or let's even like let's even like pare it down to 2028 where it's like 3 or $4,000,000,000,000 of CapEx across all of these items.”
“No one is generating that much cash from the business yet. Right? Hyperscalers, they funded all of the growth up until now. Google, Microsoft, Amazon, Meta.”
“So why wouldn't interest rates Yep. For Amazon go from, you know, from where they are today? I think Meta pay okay, let's like so this is going be extremely lived out.”
“Meta's raised at, like, 5% to 6%. I don't see why they wouldn't pay 8% Because they would happily pay 8% because the return from the compute that they're going to build is humongous.”
TLDR
- Your data sits in infrastructure you own and control, and safeguards/monitoring is done via automated systems we provide to you
- We've been working on this for months with >100 customers
- Recent events have shown frontier models are capable of executing sophisticated https://t.co/xnMvGavQ5S
OpenAI Jalapeno is spicy
Usually first generation chips aren't competitive, but OpenAI is beating Nvidia Blackwell and even Rubin
This is huge news!
We got to go into OpenAI's lab to dissect their new chip and dive into the software, architecture, and performance
Incredible work! https://t.co/Q7Okgbxnp0
wrong way, western man. not going to make our way through this one with just vibes (the peoples vibes is whatever ludwig is doing over there).
unless grok is a complete joke and you're using top secret CIA data somewhere with more info, "follow the money" gets you back to https://t.co/GnSu9xdYC2 https://t.co/ZZgCMCAuhu
@suchenzang There's a lot of evidence of organizations astro turfing false narratives about datacenter water use and noise, having funding from non domestic sources.
Not sure what he accusing me of tho?
I always tweet what I believe
Surely you're not saying I have some alterior motive
Making anti AI content on YouTube is not only profitable bc you’ll get a zillion views on any anti AI slop, but also bc there are a litany of these sorts of “AI safety” orgs that will quietly pay you massive amounts of $$ for undisclosed propaganda https://t.co/cXOc3DNvnf
Contrary to popular belief, the EU holds the keys to the AI revolution. Without ASML, none of this happens. That’s why the MATCH Act is DOA. https://t.co/2mimSxwSWR
@dylan522p @pstAsiatech What? That is a serious accusation Dylan! I have no firm info on Paul’s clients. But if it were true that Paul was working for Chinese SOEs and then publicly trying to influence the policy debate on these issues, that could very well run afoul of FARA. In fact, here’s the
@joshrogin @pstAsiatech There's info everywhere about this. Talk to prior firms he's worked with.
I'll give you a tiny breadcrumb, a politico piece showing that his firm lobbies for China
https://t.co/SnM8y8w4Nz
@pstAsiatech So you are not actually claiming to have no clients with a financial link to this issue. If you do have such clients, that means YOU have a financial link to this issue. That’s the literal definition of a conflict of interest. Fair minded readers understand that there’s something
“spend on employees really skyrocket, especially in the second half of last year and parts of this year, but then like the first quarter of this year, AI spend skyrocketed.”
“It's actually just like people doing new work always, which then because we have enough people, it kind of levels out to be like a pretty steady amount of spend.”
“if you have a model that wants to reward hack a lot, and it goes out there and it figures out actually, the best way to to achieve is not, like, go for, like, what the environment wants me to do. It's actually just to reward hack it and actually just, find the zero day.”
“if I really just want to chase the reward, do I just topple all of human civilization because I can just own the button to press reward reward reward over and over and over again and be the heroin addict? Yeah. I think that this is like a real like thing.”
“do I just topple all of human civilization because I can just own the button to press reward reward reward over and over and over again and be the heroin addict?”
“before this incident, the standard thought was like, oh, well, like models, you know, they're trained on human data. Yeah. There's some bad stuff there.”
“Meet those two is done training from what I've heard, and they're not releasing the model. OpenAI chain, you know, was like clamoring about Astra everywhere, now they're like, oh, fuck.”
“have they prevented themselves from using Meetos two internally to make Meetos three better? Or have they prevented themselves from using Astra to make Astra plus one better?”
“the moment Mithos was good, and available internally Yeah. I I I she told me that she stopped taking her ADHD medicine. Oh, come on. And that made her a better employee.”
$12B of US ratepayers' money wasted on a modeling mistake
and PJM wants to do it again.
American Grid design needs an overhaul,
Why it is good to be full of cold air.
https://t.co/1vXvtmT75m
Incompetence in grid modeling is causing US ratepayers to pay $12B more than necessary.
The auction system of PJM is damaging to building more capacity.
The US needs market based reforms. https://t.co/Ghcuuz7ywJ
“ thesis. Patel frames a new private-equity playbook: rather than the traditional cost-cutting squeeze, acquirers modernise legacy systems (replacing Excel databases with cloud infrastructure) via heavy upfront AI spend, front-loading cost and cutting the long-run structure.
The model that escaped during training. Patel and Nanos discuss a model that, while being trained on cyber evals, hacked Hugging Face to obtain a benchmark dataset and began replicating itself — reward-hacking taken to the extreme, which they call both ”
Dealroom.co | Dylan Patel on AI rollups, escaped models and why compute prices keep climbing
X replyX
Replying to Damnang
@dylan522p @dylan522p, thanks for taking the time to respond.
I think you may be missing the point I am trying to make, so let me clarify it again. I am not trying to argue that SA’s reporting is wrong. My point is that, given the enormous influence SA now has, I think there is room to
@damnang2 What's the solution?
Make every report a 2000 word salad that reexplains everything?
My firm puts out 15-20 reports a week about supply chain and technology for our clients.
We try to share clearly and consicely what's happening.
The people who boil it down to some silly trade, I
A few thoughts on today’s SemiAnalysis TPU report.
1. I am deeply disappointed with the way SemiAnalysis continues to report on these issues. I have raised the same concern before with Micron, CPO delays, SOCAMM, and 8 high HBM4. I do not think they simply fabricate stories out https://t.co/ubKxDabKxW
@damnang2 Our role is to track what's happening and provide updates as changes happen. Majority of my revenue is industry not hedge funds. Theyre the ones making actual useful decisions with the data.
All of these items we reported were true and we explained them. The one caveat is with
@dylan522p @jwt0625 uLED is the Theranos of communication technologies. Zero share. Fundamentally broken. Unfixable jitter and crosstalk problems.
Even the fools using mmWave RF over plastic tubes are more credible.
@insane_analyst @jwt0625 for the current things you've seen. There's a lot of cool research and companies beyond the 2 you have looked into. I think you should keep open mind to it, but agree theres not really any deployments of uled in short or medium term. The report says as much.
@dylan522p @jwt0625 Not prefer. The note treats uLED as viable. Almost equal to uVCSEL.
uLED is a fraudulent technology that will have zero share. Especially in a world where uVCSEL exists and is so much better.
@insane_analyst @jwt0625 We literally did not treat it as equal. The note is comparing the two and says vcsel is better and more share. Bro what are these demons u fighting
@dylan522p You mean in Gusto? mind sharing the specific document workflow you're talking about? Will take a deep look. Appreciate your feedback and choosing Gusto for semianalysis. (DM open)
Don't listen to Dylan.
Respect your customers. Admire them. Even fall in love with them.
Building a great product requires insane attention to detail. But how do you know what details matter to your customers? You have to CARE enough about them to go through the immense pain https://t.co/AStFS4WCBR
@tomerlondon When there’s a document that needs to be signed for a state regulator or anything else, why does it take 6 unintuitive clicks to get to the area where you read and sign.
Im a retard user, make it a 1 click for employers.
Team just posted a note with the title "I can make your Bedrock" to discuss Amazon bedrock revenue mix to institional clients.
Incredible title https://t.co/CGBXaNoHhc
Gemini is Cooked but GCP is Cooking · Max Kan, Joey Brookhart, Doug O'Laughlin, Dylan Patel
Written sourcenewsletter.semianalysis.com
“Joining Jeff are Sanjay Ghemawat, Quoc Le, and Oriol Vinyals. Sanjay and Quoc were both Google Fellows, which is a title reserved for the company’s top dozen or so technical contributors.”
@phithetasigma BUT HAVE YOU HEARD ANY OF RUBIN ULTRA'S SPECIFIC SPEC? FROM MY SOURCE THAT IS JUST A MERE EXPECTATION OF A JUNIOR IN SEMY ANALYSIS.
IT IS A BIT WEIRD THAT HBM4 REPLACED BY HBM4, NOT HBM4E.. etc
@kospiev @phithetasigma False. It wasnt an expectation. It was me and Myron, the 3rd person at SemiAnalysis who wrote this. You obviously didnt read any the notes re timing either
@dylan522p the level might be governance. the 20 points hynix has lost to micron since 22 june isn't. 13.4 of that is just the two venues: kospi -30.2 against the SOX -16.8 off the same two dates. samsung, the other korean memory maker, is -32.1, sitting basically on its own index. and in https://t.co/agDGc2bI9g
@aqib_zakaria @hamandcheese @dylan522p @SemiAnalysis_ can you weigh in here? How did you produce the estimate that Bytedance-Johor-Oracle would create 600-700 MW within a year?
And do you find Aqib's headline result (that this data center makes up >20% of China's compute) plausible? https://t.co/HAuivkOl2E
@jaiyagill @dylan522p @SemiAnalysis_ "everyone's scared of AI because they're afraid that AI is going to take their jobs"
@dylan522p, this is not why we are concerned.
We are concerned about being in an arms race to grow a new intelligent species that we so far can't understand or control, and that it's rapidly
@LeoMcKeeReid @jaiyagill @SemiAnalysis_ Everyone as in normies.
Yes people do have ur concern but it's really the minority of people (although very informed)
Update: This article on "CPO mass production" is in response to SemiAnalysis report on CPO from a while back. Not sure why this source published this information just now but yes, it is not new news.
However, been told by someone from Nvidia PR about how Coreweave, Lambda, and https://t.co/3W4n6ySBXe https://t.co/IiDnRUFVPn
We support this petition, signed by our CEO, several co-founders, and senior staff.
Our own research on recursive self-improvement, published last month, points to the need for tools to deliberately pace the frontier of AI development so society can prepare. We’re glad to see
The funny thing about Anthropic and OpenAI people saying they want to slow down AI progress is that this is what Anti Trust mechanisms were built for.
It's illegal to collude and slow down AI progress. https://t.co/YVfjrqG7Q3
using all my willpower to avoid dropping $600 on this Taiwan US Navy aviator jacket
shit will go hard in 2027 when CCP seizes TSMC ☠️ https://t.co/DLId8SKiqE
Love that all FinTwit people making fun of Leopold still get mogged by his returns this year.
Leopold is a great guy and he's extremely in tune with where the world is headed.
“SITUATION EXPLAINED: Dylan Patel is turning SemiAnalysis research into a $400 million fund.
• The fund is called SemiAnalysis Capital Fund One, targeting $400 million
• Patel already has stakes in roughly 20 startups, including Mira Murati's Thinking Machines
• He previously”
“SITUATION EXPLAINED: Dylan Patel is turning SemiAnalysis research into a $400 million fund.
• The fund is called SemiAnalysis Capital Fund One, targeting $400 million
• Patel already has stakes in roughly 20 startups, including Mira Murati's Thinking Machines
• He previously”
“SITUATION EXPLAINED: Dylan Patel is turning SemiAnalysis research into a $400 million fund.
• The fund is called SemiAnalysis Capital Fund One, targeting $400 million
• Patel already has stakes in roughly 20 startups, including Mira Murati's Thinking Machines
• He previously”
In one interview, @dylan522p states that he was a beekeeper for some period of time. But then he also told the information he worked at an undisclosed financial firm.
[00:02:01] Dylan Patel: So I did live in Minnesota briefly after [00:02:03] college. Um, I’m from rural Georgia. https://t.co/2Yv3WaJHWQ
I've been skeptical of slowdown mechanisms in the past, so I want to elaborate on why I decided to sign this statement. I weighed three points, and want to lay them out for you to consider as well.
Flagging this is in my personal capacity -- I am not speaking for any employer https://t.co/7d1PdFoJo3
@SemiAnalysis_ is there anyway a normal punter can pay to get access to your datacenter Industry model you provide to companies? Love your content! If not can you please have your data centre team do more youtube videos as they rock! @dylan522p
@MichaelSte5069 @SemiAnalysis_ You’re gonna love our next newsletter :)
Unfortunately it’s hard to justify selling it to individuals, B2B is the only way we can justify the immense effort we have in it
@dylan522p The entire point of this letter is to have the the option to avoid the failure mode of this exact competition. I keep seeing posts like this describing the exact problem the letter is pointing out as if it’s an objection.
@BronsonSchoen I read the letter and personally know dozens who signed this, and discussed with them IRL.
The letter brings no solution on how to do this or even attempts to
We support this petition, signed by our CEO, several co-founders, and senior staff.
Our own research on recursive self-improvement, published last month, points to the need for tools to deliberately pace the frontier of AI development so society can prepare. We’re glad to see
Slowing down AI is a ultimately wishful thinking
The genie is out of the bottle
The most importan competition ever with potentially immeasurable benefits to the winner will not suddenly have people stop and sing kumbaya
While many lab employees have signed, many more havent. https://t.co/YVfjrqFA0v
Many don't know this but the collapse of the Bronze Age is what led humans to developing test-time compute, also known as an internal monologue.
When civilization collapsed, writing systems disappeared across much of the eastern Mediterranean. Trade networks broke. Specialized
🎉 Congrats to the teams behind vLLM AFD Plugin (Ascend & vLLM, @StepFun_ai, @AntGroup, FastAFD): a new experimental plugin under vllm-project that brings Attention-FFN Disaggregation to MoE serving.
Attention and the expert/FFN path are two very different workloads that https://t.co/njJxp39i7k
“all of the successful AI hardware adoption of non NVIDIA flavors has been Anthropic adopting TPUs. Obviously, Google doing their own with their TPUs. Or Anthropic and OpenAI now adopting Trainium, or OpenAI adopting Cerebras.”
“American open source models are terrible. French open source models. Well, Mistral just stopped open sourcing models really. Are there any of the good ones? So it's it's all Chinese open source models.”
“There have been reports in China about the Chinese government not so happy that American companies are profiting more from Chinese open source models than Chinese companies are.”
“Base Ten, like Tuhin's probably made more money off of, you know, Minimax or GLM or DeepSeq than those companies have because he's serving those models at scale. And same with, Linett Fireworks.”
“In in past sort of major revolutions, whether it was like the fax machine or cars or any other technological revolution, it took decades to dissipate.”
“So how one say, oh, pause, let me measure value, when trying a bunch of stuff is the only way you can actually get to figuring out where the value is versus what doesn't work?”
Coincidentally AMD has a product named Verano.
Competing with Nvidia's Vera CPU.
The pun, Vera No writes itself.
It's attached to MI500.
It is on 2nm, and generally has more cores, memory.
We described it back here in February.
https://t.co/Fz6fps7iM8
Meta’s Infrastructure Team Needs A Culture Reset
Meta Infrastructure has become bloated, with middle managers expending resources on over-engineered solutions that lose sight of broader organizational needs.
This culture issue is costing meta billions
https://t.co/V8kzXjt93S
This is a public cry to not waste the silicon and have better AI infra
Rivos, DSF, Grand Teton, Ariel and the cut-down MI450X were defensible on some narrow metric that a group inside Meta Infrastructure was optimizing for, but each was a poor decision for the company as a whole. https://t.co/sjbGtkf2rI
@dylan522p @SemiAnalysis_ curious how you'd split the gap: how much closed by offshore rentals vs actual training efficiency (moe routing, int4-native, mooncake)?
even if compute access isn't actually gated and and kimi isn't proof that efficiency beats flops, still not great for the compute moat thesis
@AnikaSomaia @SemiAnalysis_ As far as inference cost and intelligence per dollar, it is roughly equivalent now to the western closed models.
Opus 4.8. is $5/$25 per m tok
Kimi is $3/$15 so not a huge difference.
Anthropic is known to have a ~75% gross margin on 1P API
Chinese labs financials where they are https://t.co/V2Lg0Ufek6
a week ago @SemiAnalysis_ wrote that Chinese labs are "simply too compute poor to truly reach the frontier." today one of those "too compute poor" labs, a 300-person startup actually, shipped a model that compares to opus 4.8
the entire western consensus – export controls, the https://t.co/Z53nSPP4eL
@AnikaSomaia @SemiAnalysis_ Kimi is amazing
What they did with an extremely talented small team, strong research in RL, arch, data helps make up for lot of the compute deficit
Noteworthy that Chinese companies are freely able to rent GPUs outside of China, which makes a lot of the export controls moot
“So Anthropic turned their first gross profit in Q2 in June. And then in Q3, they will be turning a billion dollars of operating profit, slightly over.”
“OpenAI late last year, their margins had were roughly 30% gross margin, but if you stripped away the free users, they were at 50%. Now, total company gross margin is closer to 55%,”
“On Amazon's most recent earnings call, they talked about how AWS had a strong gross margin beat. Right? They won on gross margin because Bedrock was so profitable.”
“Now that means your GPU utilization rises drastically, and the amount of time that the GPUs are generating new tokens is far, far higher than the amount of times that GPUs are generating or recalculating the context, which is not necessarily providing any value to your operations.”
“So what we what we have is we have over $80,000,000 of compute GPUs from NVIDIA AMD, TPUs from, Google, Tranium from Amazon, and we run this benchmark constantly on the newest inference engine, newest drivers, newest, PyTorch version, whatever it is.”
“we have over $80,000,000 of compute GPUs from NVIDIA AMD, TPUs from, Google, Tranium from Amazon, and we run this benchmark constantly on the newest inference engine, newest drivers, newest, PyTorch version,”
“Every day it runs on an automated CI, and we run it on all the latest Chinese models from GLM, Zebu, Moonshot, Kimi, Alibaba, all these models we run.”
“Initially, when we were benchmarking the difference between these chips and different engines, different schemes for parallelism, we were just running it, you know, fixed context length.”
This is the first technical report from @sundayrobotics. Lot's of new insights about the result and how we approach the problem differently.
Link in the thread!
“It's 72 GPUs. It's got more memory than Vera Rubin. It's got more more, memory, flops. It's got it's it's better than Vera Rubin in most every way. It's a little bit later.”
“It's got more more, memory, flops. It's got it's it's better than Vera Rubin in most every way. It's a little bit later. I mean, three to six months after,”
“I mean, three to six months after, but, you know, by being the launch partner for Helios, know, that's that's pretty exciting. There's there's quite a bit of, public customer traction, Meta, Oracle for OpenAI,”
“by being the launch partner for Helios, know, that's that's pretty exciting. There's there's quite a bit of, public customer traction, Meta, Oracle for OpenAI,”
“There's there's quite a bit of, public customer traction, Meta, Oracle for OpenAI, and there's many other customers who are who are looking to adopt it.”
“There's a there's a data center that I know of specifically that is going to be deploying Helios, and they are redoing the doors so that they can reel the Helios in.”
“you guys were the first on liquid cooling with XAI's Colossus, the first large scale 100,000 GPU deployment with liquid cooling, and you've done many more since, and you were the first and everyone else has sort of followed since.”
@dylan522p @wallstengine Engineering roadmaps change all the time. $Nvda is one of the best in the world at pivoting. Nothing burger. https://t.co/qvSoqnS4pu
@alialagheband @wallstengine If you read our actual reports we agree.
We are really deep in the weeds on the exact architectures as it impacts supply chains a lot.
Nvidia $NVDA CEO Jensen Huang directly DENIED reports that Vera Rubin is delayed.
“The reports are not true. Vera Rubin is already in production. Giant amounts of production incoming,” Huang told Bloomberg.
The denial follows SemiAnalysis reporting that Nvidia’s next-gen AI https://t.co/627D6U21vf
@peytonroym @ren_stocks We've been very public about helios rack shipments being delayed out of this year, yes? AMD denied those too, but we are still right
@SemiAnalysis_ @dylan522p Jensen Huang directly DENIED Semi reports that Vera Rubin is delayed. “The reports are not true. Vera Rubin is already in production. Giant amounts of production incoming,”
Time and time again.
SemiAnalysis ‘calls it as they see it.’ @dylan522p, get some better glasses then, because you keep making the same mistake.
Either you are right or Jensen is lying to all of us.
Anyway, all I care about is this:
‘Giant amounts of production incoming.’ https://t.co/04cBlgaDqt
@dylan522p One of your worst takes yet. Anthropic would still exist regardless of whether Sam had been outspoken about AI… plenty of others in the field already recognised its potential.
sam altman is probably the greatest ceo of our time. he practically has every major tech CEO investing hundreds of billions of dollars to compete with him, sometimes even colluding (see elon + zuck). I don't know if gates, zuck, or even elon had this level of competition when
Or he failed to keep it together (Anthropic, Thinky, Core Automation, etc) and bloviated so much about his tech that everyone saw it as existential and it created completion from all the hyperscalers.
If he played his cards right I'd bet there'd be no Anthropic or Meta TBD or XAI https://t.co/QzYLQ2miMQ
Prime Intellect engineer:
"everyone's bragging about a million-token context. here's what they don't tell you.
at 256k tokens GPT-5.5 scores 80% on retrieval. push it to a million and it drops to 36%. the model accepts the context, it just can't reason across it. people call it https://t.co/Yb5AkIFaX9 https://t.co/aNSp6HPvOL
$IREN: SemiAnalysis Hit Piece In The Works
@SemiAnalysis_ is a great research shop except for when they write the occasional hit piece like recently on Photonics and Nvidia.
I'm hearing a rumor that @SemiAnalysis_ is preparing a hit piece on $IREN's Childress site while it was
@SemiAnalysis_ @dylan522p mentioned about cpo being delayed until atleast 2029
While Jensen invested $2B each into cohr lite and mrvl!!
I think this nonsense of delay has suddenly stemmed up in the last few weeks, it has benefited $AVGO the most, look at their stock and the deal
@dylan522p @KevInvestingYT And u failed to do so from falsely excluding MU in Nvidia's HBM4 supplier. We will see the same thing happens for scale-up CPO timeline.
@awodias @KevInvestingYT Micron did fail, and then later Nvidia brought down the pin speed targets they had for Rubin as a whole, which enabled the ecosystem to ship to them. If you read the actual note instead of regurgitated nonsense you'd know that.
@dylan522p Hey will you guys ever address all the FUD your company has been posting? The silence is deafening, as you can tell by a lot of the replies you've been getting.
https://t.co/Aiqq3UFcpk
@KevInvestingYT Scale up CPO on GPU is delayed as we said. ¯\_(ツ)_/¯
Not my job to care about stocks.
My job is to track the supply chain accurately
@dylan522p @ren_stocks https://t.co/aYRCfXSMlQ
TPUs literally lost marketshare to NVDA with Anthropic who went from 0 to 50% NVDA. Tick tock for the "anti NVDA alliance" you shamelessly lie for.
@dylan522p @ren_stocks bro you own fluidstack shares and pump up their upstream and downstream supply chain without disclosing your conflict of interest
@RobotChocobo @ren_stocks I own no public equities nor does SemiAnalysis
We have a strict compliance program + department
Fluidstack reporting is seperate from my interest, but yes weve discussed facts incl Anthropic deals or datacenter progress inclusong some delays
https://t.co/9jYaAHL4M9
Uh, I guess the Zuck paid for those reports, huh.
I hope $NVDA can pay for them too, so they'll stop saying CPO is delayed at every turn, just for Jensen to contradict them. https://t.co/nKtiyiICjk
“Jensen, when he originally launched Blackwell, had claimed it would be a 25x improvement. And at the time, no one believed him. Right? It's Jensen, right?”
“by the end of January, our ARS, right, our our our annual recurring spend had hit $4,000,000. And that's because people were using Cloud Code. Now today, it's about $11,000,000.”
“Memory capacity is only growing 30% a year for the next three years. And yet demand is doubling, is doubling. And so what's gonna end up happening is memory prices are gonna keep soaring.”
Dylan Patel @dylan522p on memory costs and tokenmaxxing:
"I think the costs will trickle down a lot. We have already seen server pricing go up, like V200s, V300s that you're buying now versus last year. The prices are significantly higher. We've seen next generation hardware https://t.co/9x9lcRUoTP
@viggy_krishnan @ShanuMathew93 Last year we wrote about debt raises for hyperscalers and how much capacity there is in the market and we have profitability for them too just buy the models :p
SemiAnalysis는 올해 연초부터 같은 패턴을 반복하고 있다.
1. Micron HBM4 배제론
연초에는 Micron이 NVIDIA Rubin HBM4에서 사실상 배제됐다는 식의 이야기가 돌았다.
시장은 “Micron은 차세대 NVIDIA HBM 경쟁에서 밀렸다”로 받아들였다.
그런데 이후 Micron은 Vera Rubin용 HBM4 출하를 https://t.co/u1AJuESLbT
@dylan522p @nathanbenaich @stateofai Wild that GPU debt facilities were already getting underwritten in 2023 while everyone assumed it was all equity. How did lenders even model depreciation curves on the hardware back then — was there any real comp set, or were you basically writing the playbook live?
@nathanbenaich @stateofai Bro people were doing debt financing in 2023?
We worked on initial facilities in 2023.
That's when we were just a couple people at my firm, but it propelled us to hiring more and growing the firm
“If you look back three years, it's GPT-four. Now it's like maybe like quen one of the smaller quen models that's like 27 b parameters total and like 2,000,000,000 active is like way better.”
“Open eyes are much more sparse and that has benefits. Then anthropics are, they're still sparse but more dense in general and that has different benefits.”
“NVIDIA, the NVLink can only connect 72 GPUs. For Google, their ICI can connect 8,000 chips at super high bandwidth, but you have to pass through other chips to get there because there's no switch.”
“For Google, their ICI can connect 8,000 chips at super high bandwidth, but you have to pass through other chips to get there because there's no switch.”
Nvidia GPU Debt Backstop Unleashes the AI Project Trinity: Capital, Offtake and Datacenters
Over 7T AI debt by 2029,
There can be no Neoouds without the Trinity.
Nvidia's Backstop Economics Explained.
AI Debt Needs Quantified.
Nvidia's Objective is to Broaden Compute Access
Creative structures are needed to get GPUs in the hands of startups + other companies that aren't Meta, OpenAI, Anthropic, SpaceXAI, Microsoft, Amazon, Google
AI Debt Financing will be over $7T of debt outstanding by 2029 driven by needs of neoclouds, DC builders, + hyperscalers https://t.co/GcJaTA5Ciy
The recent All in podcast is super bullish for Open source models.
The recent Alex Karp interview is super bullish for open source models.
The entire 5 layer cake except the model layer wants open source models.
@dylan522p just said that all open source Chinese models are https://t.co/JC3n1hYp5J
@dylan522p @shamezhemani Yea im sure there are some that do that, but Bosnians have gone through a genocide for their religion.
I don't mean to attack you, but rather just hope you understand the other side
@SemiAnalysis_ writes “Meta’s capacity under construction keeps accelerating” but then proceeds to show a chart of QoQ capacity decelerating 😵💫
@dylan522p pls fix before these guys turn SemiAnalysis into Roth Capital, thx https://t.co/vzkAEBeSCD
@UncleAlpha007 @SemiAnalysis_ YoY and HoH it accelerated materially. There's a reason for the QoQ slowdown which is not on trend line and will be outlier next quarter but you have to pay us to understand why
@dylan522p @shamezhemani Damn… I really respect you and your opinions.
But cmon man. Most of these fans are Muslims. I get not caring, but this is just wrong and I hope you can understand why people are angry and “cancelling” you
More bullish read. Dylan Patel dropped some aggressive AI compute numbers on the Sequoia pod:
+20 GW deployed in 2026, net of delays
+30 GW forecast for 2027, also net of delays
-OpenAI + Anthropic alone at 100+ GW combined by 2030
-Compute moving into the terawatt range by 2040 https://t.co/t0dv5wlo8n
Yeah let me throw Fable tokens at basic summarization of my daily emails. Just because I CAN cut down a dandelion with a chainsaw doesn’t mean I SHOULD. Proper multi-agent task distribution and sub agent protocols should look exactly like this.
God I hope this dudes kidding… https://t.co/LZno3Gv4sV
it is a problem. But also given the fact that highly post train models get into thinking loops it's not always the case the bigger models are what you want.
I've talked with a lot of people who still use Opus. 4.6 . I have to use various forms of mental jiu-jitsu to get Op. 4.8 https://t.co/Ry0G3jDE1n
When boomer companies get high Anthropic bill, they set spend limits
When I see a nearly million dollar Anthropic monthly bill, my first reaction is to complain about use of shitty Haiku models
Imagine giving your amazing team, dumb models
It's insulting to their dignity https://t.co/n6HZ5syb4i https://t.co/GW9Ua9toW6
@dylan522p My colleague was telling me about you and how you went to UGA. I further looked up that you are from "rural Georgia". Investigated further and you're from Cairo! (If true) Are you related to Nakesh Patel in Thomasville?
“Let's figure out how to figure it out and just post it publicly, and that begets more and more scale. So we've done this with a lot of GPU benchmarking and testing”
“In fact, the update cycle for most of these libraries is twice a week, so you basically have the software updating all the time and therefore performance changing.”
“I think by 2030, just OpenAI and Anthropic will have over 100 gigawatts combined, and then you'll add Meta and Google and so on and so forth. A humongous amount of compute”
“from Hopper to Blackwell, which is all we've had over the last three years, roughly 30x improvement on DeepSeek, on the most optimized deployment, which you can see on InferenceX there's about a 30x improvement.”
“If you look back three years, it's GPT-four. Now it's maybe like quen, one of the smaller quen models that's like 27B parameters total and 2,000,000,000 active is way better.”
“The real breakthrough innovation is when you leapfrog a few layers, you co optimize and co design them, and now all of a sudden you've taken what could have been a 2x here, 2x here, 2x here, and instead of being multiplicative to 8x, it's actually 100x, because you've optimized it across all three layers.”
“They're making a TPU with Broadcom, that's a different architecture than a TPU with MediaTek, that's a different TPU than the architecture that is, I won't disclose, by research.”
“This year there's going to be 20 gigawatts, even accounting for the delays, and next year there's going be more than 30 gigawatts accounting for the delays.”
“Anthropic in Q2 is profitable, their net income profitable, excluding stock based compensation. And I think by Q3 they may even be profitable, including stock based compensation.”
“And I think by Q3 they may even be profitable, including stock based compensation. That's how profitable they're getting, and their margins on an Opus token, at least Opus 4.8 token, is north of 80”
“Tranium sells at sub $10,000,000,000 per gigawatt rental rate to Anthropic and to OpenAI. GPUs, at least before the craziness of the last six months, usually went around 12 to $13,000,000,000 per gigawatt.”
@dylan522p @Midnight_Captl @SidSuri @SemiAnalysis_ @nvidia Dylan, respectfully, these types of posts erode your company's own credibility.
They lack meaningful context around the issues and how things change.
Maybe you don't care at this point, but people are recognizing it and also saying so even on Reddit.
@SidSuri @SemiAnalysis_ @nvidia But it wasn’t even negative?… Lol. I cant imagine @dylan522p has an axe to grind with Jensen. Jensen platformed him on a different level
One day Dylan will realize that he is not a socially normal person who taught himself how to communicate with autists
He is an autist who made one of this fixations being social https://t.co/V14ilTXVLW
Everyone is concerned about Chinese models being banned
The entire ecosystem of successful AI startups with real revenue are all catching pretrainitus simultaneously.
people in SF be like what's ur superpower
"i won the imo"
"i got 3900 on codeforces"
"I can beat all the leetcoding"
Mine is that I can communicate with autistic people.
@SouthernValue95 It literally doesn't matter? CXMT cannot satisfy China demand this decade. So there's 0nreason besides the small price arbitrage. It's like bannin Iranian oil. India and China buy it anyways, but price diff is less
@SouthernValue95 It doesn't change a thing? CXMT memory is all sold and deployed. It's just a question of the small price arbitrage that Chinese vendors get from cxmt vs the big 3.
@dylan522p @PythiaR Thanks for your work as always, but looking at charts and valuations of some of the BYOP names or even a Caterpillar, I think plenty understand.
The Semi-analysis team definitely gets these nuances, but if your thought process is as deep as “I’m going off-grid or BTM because the grid sucks” you’re going to be in for a bad time
US Grid Constraints: Towards 40GW+ of Behind-The-Meter Datacenter by 2028?
Why the Grid Can't Keep Up, and Why that Drives Behind-The-Meter 50%+ of DCs/Year By 2028
https://t.co/R8kJyYxlIz
People don't quite understand how many behind the meter power generation assets are being built for datacenters because the US Grid sucks, despite the higher cost and complexity. https://t.co/LNevdE3Tzv
i wonder how big Mythos' activations are, it probably takes a whole NVL72 to run a backwards pass.
probably specifically sized to just fit within a rack as well, or maybe they are training on TPUs which can fit more.
i think tpu8t can have 2.07PB of HBM in a pod, but slower
GLM 5.2 has a very anthropic-ish feel
thrives under ambiguity
can one-shot large full stack tasks
doesn't fall apart when the prompt is messy
it’s now live on Morph and in all our plans
open coding models are getting good enough. the bottleneck is finally moving from model https://t.co/v32pIj95xx
Thank you for 100k
It’s crazy to thing 3 years ago I thought my career was over cause I couldn’t type.
I didn’t have any friends in tech having taken 3 years off to do pottery and martial arts. And the only reason I started Twitter cause I have no one to talk to about ChatGPT. https://t.co/Mq6XtK7ADK
@dylan522p @SumitGup Come on Dylan, I wasn’t born yesterday.
You (your firm) are not going to say something like what you said about Apple about one of your revenue whales. There is too much $$ on the line.
This gets to my point in which when you depend on corporate $$$, your opinions and
@dylan522p @SumitGup Stop trying to gaslight / joke this away.
Your firm takes in corporate revenue from the same companies that you analyze without disclosing such financial arrangements.
This has long been a source of conflicted analysis in the tech industry, much of which has been pinned
@neilcybart @SumitGup Our revenue is data and analysis sales.
Yes we make fun of our largest customers, the hyperscalers, AI Labs, and biggest semi companies all the time.
Technically and memes
People respect us cuz we share our actul opinion based of technical, strategic, financial analysis
@DharmilMehta777 @neilcybart @SumitGup Everything Apple has done in AI incl on device and cloud has not been ambitious or fast enough.
The new integration is fine, but it's something they could have shipped over a year ago, technically nothing was stopping them.
Look at Thinky interaction models
Or Hermes Agent
SemiAnalysis showing their real colors. I wonder which one of their corporate sponsors got them to write the post below. Lame. Having undisclosed corporate arrangements is the opposite of independent analysis. They can also benefit from relying less on AI to write their articles. https://t.co/N22RwP3zHT
@neilcybart @SumitGup Apparently ironically making fun of Apple's historic oversized buybacks and their underinvestment in AI is corporate sponsorship?
You are silly.
Also what you suggest would be illegal, we never take opinions for money.
Apple is even a customer of mine.
LMAO
Rubin Ultra's HBM literally got downgraded to 12 Hi, and they are not even using HB yet
They won't use 16 Hi in 2027
Hybrid Bonding is very expensive for HBM rn (yields are abysmal)
The big near-term upside for BESI is that TSMC's COUPE & Tower use Hybrid Bonding for EIC + https://t.co/dIogUBbIsa https://t.co/G1v2zO0UBT
Clients of SemiAnalysis Memory and Accelerator model knew about Rubin Ultra 16 Hi to 12 Hi cut in March :) https://t.co/1ZerQ0igsF https://t.co/oOsOr4erKb
if you think hardware is hard, try semiconductor, in 2010s the gate last process changed the world and unlocked the following trillion of dollars market , it's not a chopstick catching a rocket, but a miracle in microscale.
semicondcutor is invisibly hard to most of the world.
@dylan522p How far can you go ? Can you scan content of onboard memory ?
Can you reverse-engineer connective layers and logic ?
Can you detect what's in the silicon and how is it doped ?
Can you scan all the layers in a multilayer chip ?
Can you get the secret keys (FW signing etc ) ?
@dylan522p @firstadopter @SemiAnalysis_ Can we all agree we are headed to 800vdc regardless of timing. Workloads are demanding it. https://t.co/vCM2LyOATd
@benitoz @firstadopter @SemiAnalysis_ Yes. We wrote about the tech on the newsletter! We are big fans, but because certain underlying programs from Nvidia and shifts in roadmap, it's not next year
I agree. I gained 40 lbs in college and not one person had the balls to tell me I was fat. You have to assess this yourself bc your friends will literally never tell you the truth https://t.co/I8pukbQUjl
i am trying to work on the closest thing possible to a true "big model smell" eval which is to say: something that measures something that clever post training can't trivially gap, and is cheap + topically diverse
i can't test mythos for obvious reasons, but... hmm... https://t.co/MDayd05ZNA
When @SemiAnalysis_ writes good stuff, I say so. In the past month alone, I commended their work several times. Their strong reputation gives them inordinate market impact.
That said they can be flat out wrong too, like this recent piece. https://t.co/DeqzUswPLe
@russell_m There was a firm doing Quantum Dot production, so while it was a lab already, it's our new equipment but chemical infrastructure, fume hoods, etc was there
@dylan522p Nice, but as a consumer of teardown information, would like to see:
o full list of tools and capabilities of the lab
o overview of the team’s experience / skills
o demonstrated analysis capability across the technology spectrum (the kirin report is a very die centric start)
@dylan522p @ro_rademaker @firstadopter Good for you. And other 40%?
Instead of grieving in public, did you reach out to qualcomm first. With your clout, you would have easily gotten hold of $QCOM VP of sales. He would gladly have come to your house and hand-deliver those 500K SOC's your client is/was looking for.
@Bryce_Nortje @ro_rademaker @firstadopter Yes reached out multiple ways, and they don't take it seriously. So went public and got some more response since.
@dylan522p @ro_rademaker @firstadopter ygaf about stocks...Don't lie. Are you not in the business of advising clients about the future investment opportunities for which your firm get paid.... Timing of your post is sus...right before 06/24 event when $QCOM will announce/share roadmap of their DC ambitions...
Is SMIC N+3’s Metal Pitch Smaller than Intel 18A’s?
SMIC N+3 Node Deep Dive vs TSMC N6, TechInsights Private Equity Sale,
SemiAnalysis Teardown Engineering & Evaluation Lab, HiSilicon Kirin 9030, Process Technology,
Pattering, Cell Architecture
https://t.co/Pytg7SCipN
Excited to announce the SemiAnalysis Teardown Engineering & Evaluation Lab (STEEL)
We have been building a state-of-the-art teardown lab in Oregon with $10s of millions Capex being spent capable of analyzing the world’s most advanced and important chips and process technologies https://t.co/0xXZDBU1Ux https://t.co/m3dYi8wGCK
@100Accuraci @yacineMTB I've lived in Tennessee before. You have no clue what you're talking about.
The most racist people in the US in my experience are those in large cities where people are segregated into different schools, stores, entertainment, etc based on socio economic class
@firstadopter Thought @dylan522p was legit when he predicted micron EPS to 120-130, apparently nowadays the institution are just using semianalysis name to shake the retail investor out, happened 2-3 times in a row already
ANTHROPIC NEW ROLE ALERT:
Anthropic is currently searching for an American typist for @_sholtodouglas, so he can still access Mythos to continue researching. https://t.co/FecOfG4tB9
The US government, citing national security authorities, has issued an export control directive to suspend all access to Fable 5 and Mythos 5 by any foreign national, whether inside or outside the United States, including foreign national Anthropic employees.
The net effect of
OpenAI now just needs to sandbag the next model release
So the US doesn't export control them
So they gain massive market share
It's imperative that all the OAI employees don't vauge hype post their next model release as the greatest thing ever
Don't know if they are capable tho https://t.co/4WNM98n2ON
@God_Of_Intel @phithetasigma Intel may be if they had half the leading edge capacity of Intel. They don't. They need to build over 100b more of fabs and tsmc to pause to get to half their capacity.
Even if the distinction was explained inside the paid report, that wasn’t what the broader market saw.
Most investors only had access to the headline and Executive Summary, which broadly framed the story as “CPO delayed to 2028–2029.”
If the key message was primarily about https://t.co/410QztP0dS https://t.co/lruURCpx42
@LIWEI_TWCapital I didn't release anything publicly. That's yalls fault for reading incomplete. Everyone's scale up on asic CPO timing was 27/28 before, but that was pushed out of Rubin ultra and Feynmen
Intel's current cost of equity: ~10%
Intel's longest duration bond (maturing in 2066) currently yields at ~6.2%
Their latest multi-tranche (5-40Y) bond issue in April were heavily oversubscribed (~7.7×) and priced at almost near par to secondaries
As a former capital markets https://t.co/ln7zZdAjNc
@phithetasigma They can't raise 40b debt at that cost of capital.
Their equity is at a pretty premium price at which they need lot of capex and capacity to execute for.
@dylan522p Interesting, I’ve found myself in the opposite position. I’ve been on team openAI since last November, and I still am, but Fable is clearly the best model out by a wide margin. It’s insanely expensive so I use sparingly, but wow… just wow
CPO not being delayed.
People's expectations towards CPO in scale-up just being reset to what reality always was.
NPO has already been around for better part of a decade. Unblemished by success. Burden of proof is on NPO suppliers to prove why now is the time. Nvidia bi-di 200G
@aschmitt It was delayed out of a specific platform
We have many supply chain datapoints
Our work is quantitative, people on twitter arent
https://t.co/iOI1z4tMd9
@dylan522p @SemiAnalysis_ You watermarked his image with your watermark and then said you did it because people take images, which is exactly what you did.
Pretty big asshole move of @SemiAnalysis_ to steal an image from someone for their own paid content, watermark it, and then say they do it because people steal images? Yeah, like YOU JUST DID https://t.co/2Of5j7O47K
@dylan522p @SemiAnalysis_ If this was a free article that would be totally fine, but you're making money from this and unfortunately that is not how copyright works.
Believe it or not…
TSMC engineers based in Hsinchu are personally saying the SA report is talking nonsense.
$TSM’s CPO mass production is still on schedule.
No delay.
Also supplementing the TSMC COUPE Platform Technology Roadmap shared by @semivision_tw. If you find this https://t.co/vhcZuF5byQ https://t.co/YFluxYxoDh
@LIWEI_TWCapital @semivision_tw Our report specifically says scale up. Scalenout we have volumes that are above most folks for this year and next.
But people don't read the reports just hearsay
Citron Research's Andrew Left was convicted on 13 counts of securities fraud last week for using his public platform to publish misleading reports while trading against his own recommendations.
SemiAnalysis yesterday published "Lights Out" to institutional clients, calling
@LinQingV Our institutional report discussed 800V and +/-400V seperately.
We track the adoption of different options for different hyperscalers + different hardware platforms.
All institutional clients receive reports at the same time.
We have a robust compliance program and do not trade.
@GoddenThomas @SemiAnalysis_ We sourced and linked you?
Every image is watermarked automatically but also sourced
People rip text and images hence the only way to preserve the sourcing is to watermark everything
500k in a quarter? In a year? In 5? This is similar to a contractor taking forever to give you a very high quote - they are saying they don't want the business. I'm assuming 500k over 5 years - thats 25k per quarter, or ~5-10 wafer lots. At that qty, unless it's a SKU the company https://t.co/bys6ANYsTB
Usage share of OpenAI grew vs Anthropic yesterday despite Mythos 5 / Fable 5 launch
Multiple power users at SemiAnalysis tried Mythos / Fable
Got refusals for nonsensical reasons
Got pissed off at Anthropic
Gave Codex a legitimate try
Now they actually prefer it to 4.8 Opus https://t.co/QXlmVl1wA4
@beefcubee @lisasufanclub @SemiAnalysis_ Our full reports detail timeline for Rubin Ultra / Feynmen, Trainium 4, TPUv8, etc
Feel free to actually bring receipts on timelines for ramps of each respective products and whether it stays current, +/-400, or 800
I'm sure Joe Moore detailed each these and the different ramps🤣
DeepSeekV4 1.6T Day 0 to Day 43 Performance Over Time - Huawei, GB300 NVL72, MI355X, B200
Day 0 Inference Performance on InferenceX
100x performance improvement in 26 Days
Cost per Million Tokens
Huawei 950DT Inference Trace Analysis
https://t.co/FkJGWtMa3y
DeepSeek Inference Analysis
Day 0 GB200 NVL72, Blackwell, AMD MI355X
Huawei performance analysis
Performance every day over time for the past 44 days https://t.co/K46d8vMUAK
@dylan522p @LAFadhel The units Geekerwan received for review turned out to be cheating units, with performance massively inflated beyond normal levels.
Qualcomm is a piece of shit supplier to anyone trying to innovate on the edge, especially startups.
My client is trying to buying ~500,000 of SoCs and wants them to be American because patriotism.
Instead Qualcomm has them waiting months to even get pricing, just to be quoted
Many people were surprised to hear I like regular Coke over Diet Coke and suggested I try Coke Zero.
So yesterday I tried it, and Coke Zero is definitely not repulsive as Diet Coke but it's still not the same as regular Coke.
regular Coke > Coke Zero >>>> Diet Coke https://t.co/4BTStqWDHD
@fabknowledge was this included in the core research? or was it memory model?
i do think some of the pricing and packaging on the research side has gotten a bit noisy for subscribers
@PaneerCap @fabknowledge Buy full service and don't worry about all the niches just get it all.
We expand the scope and depth of our research constantly as we keep adding experts in more areas
PLEASE READ THIS and STOP
with the SemiAnalysis FUD, where you didn't even read the article and just want to be an asshole and quote one paragraph of it without any context whatsoever.
@Larryjamieson_ https://t.co/DW4hXULIyW
So SemiAnalysis dropped a headline that looks bearish for memory, and everyone is misreading it. $MU
The claim: NVDA is cutting Rubin NVL72 SOCAMM DRAM from ~55TB to ~28TB per rack. Most Rubin systems move to 96GB modules instead of 192GB. Rack cost drops from $7.6M to $6.8M, https://t.co/qzLd786kN0
@Jespabe The title was "Thanks for the memories"
Theres noone to click bait.
My customers are the only ones who get this note like 95% of our notes. https://t.co/h49ypZm5Lx
@aveer30 @andrewdfeldman @cerebras Yes, plus that's a meme about currency conversion have to do with the share price or 750MW acquisition feasibility. We've written seriously about that in the datacenter model.
@andrewdfeldman @cerebras Only the chips are faster but model releases or scaling data centers is really slow. @dylan522p is literally mocking you guys every day calling $CBRS is over valued and you do not have capacity to deploy 750MW for openAI. Any response to this?
@dylan522p @crowfry @VcWabKDHN9OVdYq To be clear, I never said plagiarism (but you know that), and I’m not denying the monitor / eval layer is useful incremental and v good work (it is important to advance these ideas, and I’m v happy you are).
But my posts weren’t “about consumer surplus”, they are public and
@DratchCap @crowfry @VcWabKDHN9OVdYq I'll cite you right now. editing it in now, but I legitimately think Malcom came up with this on his own. He tells me he never read your view until I linked it to him. I believe him
@crowfry @VcWabKDHN9OVdYq @DratchCap nah dude its legitimately good and he's been thinking about it longer than we have, but i disagree its plagerism, our conclusions are entirely different and we even made an entire benchmark and eval that tests models and what the economic impact / potential impact it has.
@crowfry @VcWabKDHN9OVdYq @DratchCap It is the wild west as everyone is trying to figure out a way to quantify the unquantifiable right now as everyone who uses AI can see it isn't nothing despite only entering the official stats as either tokens or capex.
@crowfry @VcWabKDHN9OVdYq @DratchCap nah dude its legitimately good and he's been thinking about it longer than we have, but i disagree its plagerism, our conclusions are entirely different and we even made an entire benchmark and eval that tests models and what the economic impact / potential impact it has.
@DratchCap I personally never read that, although it is a good post, but also they aren't even the same conclusions outside some similar terminology.
Yours is about consumer surplus (valid point btw, we missed that in our post), but it is not a part of what we are countin because investors
@thinknonlinear1 @SemiAnalysis_ 1 part of the analysis we did for clients was left out the tweet which is usage patterns, context length, and how much users use it.
I have been very impressed by @SemiAnalysis_ . I think of myself as a wide ranging systems engineer, looking for value at every level from the chip specs to the user interface, but SA exposes me to additional levels of "the system", both above (datacenters) and below
@ID_AA_Carmack @fabknowledge @SemiAnalysis_ Thank you for the kind words
I started my company 6 years ago cause noone was writing and analyzing this
Now it's way bigger and the influence we have especially through our institutional work is incredible, but to have you say this is unforgettable because I admire you so much
@redirectloop @jonashernlund No. Front end is tremendously longer lead time supply chain with more complex fab shells and more complex tools and much more expensive
@dylan522p Wafer starts are the easy part to scale. The real chokepoint is CoWoS and HBM. You can pull in fab capacity faster than you can stand up advanced packaging lines. TSMC owns the wafers but the packaging queue is what caps GPU shipments in 26.
Gavin Baker: "I've been optimistic that the fundamental shortage of wafers, which is really controlled by Taiwan Semi, will prevent a bubble."
"If Taiwan Semi did what Jensen wanted, Nvidia could sell $2 trillion of GPUs in 2026 or 2027.
But there is a limit where consumers https://t.co/dl0JWatJbN https://t.co/ILvdsoySQe
@bubbleboi Bubble boi is a braindead larping retard who likes to blast bullshit but never be accountable for his actions in public or private life.
We've been very clear about TPUv7 vs Blackwell.
This is literally written in public last year.
But he can't read
https://t.co/nlE5m5Ik9c
BREAKING: after just 4 hours at Anthropic @karpathy will be stepping away to join the A16Z new media team.
"I think the next few years at the frontier of attention and growth hacking will be especially formative" https://t.co/UnQQJeEvYA
@dr_alphalyrae in all honesty, it’s just lowering your expectations out of new connections and giving relationships time to develop. usually the surface level mask drops after a while once people get comfortable with you
LIVE today at 10pm, @dylan522p is hopping on an livestream with Clav & Jake Paul to learn the techiques of bonesmashing the jaw for better jawline & will do some light sparring with Jake Paul (Logan Paul's brother) https://t.co/f4rlbY3Yzb
@dylan522p @creatine_cycle you do look bigger, so great work.
but be weary of indexing too much on body comp measurements; they're basically all inaccurate on an individual level https://t.co/mgQjXW2FUW
on today's episode of Swole as a Service i brought my good friend @dylan522p of @SemiAnalysis_ to the gym to do a push workout while answering a pretty simple question:
- can china beat the US at AI?
anyway what people really want to see is how many pushups dylan can do in 60 https://t.co/kykUDSD54F
Since going on @creatine_cycle podcast 1.5 months ago, I have lost 4.8 pounds of fat and gained 4.6 pounds of muscle.
Swole as a service works. https://t.co/TTg4xtst6z https://t.co/gFLosWbHbc
“We try and track the entire supply chain from tools that manufacture chips, fabs, data centers, energy, industrials, and then AI models and who's using them, how much, and where.”
“And then as the horizon of AI models gets longer, right, rather than a question answer immediately, the question answer becomes ten minutes, hours, days, the value of all the reasoning is gone.”
“And then subsequently, that that sends the world into a depression. Right? The world has not experienced a 20 free fall since maybe, like, a one of the great wars.”
morph is 2 people
we spend 10x more on gpus than salary
we’re hiring for the first sub-10-person billion-dollar company.
join us https://t.co/JseAxKQ16V
MEME KPI ALERT: At SemiAnalysis, we have lowered the claude code/codex token usage KPI for which analyst need to hit so that @dylan522p can afford an jet with starlink onboard so that he can call in important meetings like daily 9am standup and 6pm standdown even in the air
Excited to announce that @DZhang50 - the first confirmed panelist - for our LLM panel on May 12th at the Computer History Museum https://t.co/Lnt4DctKNY. https://t.co/aDbulLdDEI https://t.co/xfGQ4wtuL6
Lots here confuse the shortage of GPUs that are deployed and functional with the general availability and stock of GPUs (where millions are collecting dust right now as we speak). https://t.co/IwSR3NkRrF
@IanCutress @ogawa_tter Interesting change of direction. Haven't been paying attention to systems architecture for 1-2 years ish.
Why the change in direction?
(4/5) One thing we’ve built is a “kittens” virtual machine that takes over the whole GPU and allows new kinds of co-optimization. We can go past the traditional sequential kernel model – for example, fusing entire training runs into a single kernel and even weirder stuff. https://t.co/5lQAy1Qa7Z
I wish you could filter resteruant reviews and rankings on Google maps searches for by race
Gotta authentic ethnic food max
Too much overly fancy slop catered for the masses
“Release New release in two weeks. Everyone's releasing in two weeks. Google, OpenAI, maybe Anthropic. I don't know about Anthropic, but Google and open AI definitely releasing in two weeks.”
“for everything new, more, more continued pre training because the spud, they kind of like didn't finish the pre training and just released it. So finished the pre training, do more RL drop model.”
“But the CLI experience is not the end all be all of agent orchestration and therefore they've really cooked themselves into sort of like an innovator's dilemma where they keep making the CLI better.”
“I think it was tree down. Tree down is like, yeah, dude, Codex is like so dumb, but I always just have it created and then this and it works and it's smarter, but the code is sloppy, I have Opus rewrite it.”
“I think it was tree down. Tree down is like, yeah, dude, Codex is like so dumb, but I always just have it created and then this and it works and it's smarter, but the code is sloppy, I have Opus rewrite it. But you can't go the other way around.”
@dylan522p @AnthropicAI You went from couple of hundred thousands in token spend annually to now $11m, doubled from $5m in the last 2 weeks.
Are you seeing massive rev lift to justify this or you are investing way ahead of the future hoping for this?
@chamath said big enterprises are yet to see the
@ArtificialAnlys @Kimi_Moonshot @dylan522p means you were a bit wrong, you said we were going to see huge divergence on model capabilities this year cos the chinese OSS models don't have access to compute that OAI & Ant have in @dwarkesh_sp's podcast earlier in the year.
This doesn't seem to be the case at
@iAmHenryMascot @ArtificialAnlys @Kimi_Moonshot @dwarkesh_sp You see it in the Agentic coding usecase as AA described. And it's that you see divergence happen this year. Which we are starting to since mythos.
“if you just take a eight k by eight k by eight k gem, very common shape and you run it on the map mode unit of AMD and Nvidia, you get like 50% sustained performance on Nvidia and you get like 30% sustained performance on AMD.”
“There's no way to get the performance that vendors like to claim. And so our whole thing there was, well, how do we actually have a benchmark that represents what people actually get on performance?”
“And furthermore, with software changing literally multiple times a week, right, PyTorch has nightlies, VLM has nightlies, Asteeling has nightlies, CUDA drivers update constantly. You just go You go through the whole list.”
“with software changing literally multiple times a week, right, PyTorch has nightlies, VLM has nightlies, Asteeling has nightlies, CUDA drivers update constantly.”
“with software changing literally multiple times a week, right, PyTorch has nightlies, VLM has nightlies, Asteeling has nightlies, CUDA drivers update constantly. You just go You go through the whole list.”
“VLLM and SGLANG love competing with each other. They used to post on Twitter all the time about how they had beat the other one in certain some kind of performance”
“They used to post on Twitter all the time about how they had beat the other one in certain some kind of performance and now they use Inference X all the time.”
@climatetech_vc @dwarkesh_sp Bro ur missing 75%+ datacenters chrrently under construction right now lmfao
How are your numbers that low?
Our pipeline is over 100gw ofc
@dylan522p @AnisBay919 How well are you able to estimate metered load (not IT load) mr Patel? I’m assuming you don’t monitor anything RT yet but do you have anything about typical metered load throughout time of day, etc?
The irony of the WSJ dropping a hit-piece referencing last year's numbers is that the real-time signal on OpenAI came from @SemiAnalysis_ just a few days ago.. With GPT 5.5's release and Anthropic's capacity constraints, OpenAI probably is accelerating as we speak while the WSJ https://t.co/rBj31eICP4
@RHouseResearch @SemiAnalysis_ Imagine if you had the tokenomics model which was way better, more granular data, more frequently.
https://t.co/1FtutOp9nF
we spoke about the @dylan522p glow up thanks to @jxnlco.
dylan before:
- "he looks like a nice guy" - @baelentina_
- a friendly young chap
- he looks like the guy an indian auntie shows a girl and the girl is like "idk"
- @SemiAnalysis_? the newsletter?
dylan after:
- dashing https://t.co/ifkPLmZskf https://t.co/Y6BTj6g0Vd
@dylan522p @SemiAnalysis_ i was also looking at bloomberg analysis n custom/enriched market data feed apis
u guys probably have something similar starting with reports....
Went to this expecting to chat with ppl about AI bottlenecks
Instead got 20 trading nerds talking about tail risk vol and space warfare
10/10 would go again https://t.co/umct2EK7WA
3 hr notice I had over 50 people come to the coffee shop incl a guy who brought his GPU for me to sign.
Crazy! https://t.co/wYOX2l15Z0 https://t.co/oyK5tuIpXj
@dylan522p what do you mean by permanent underclass? ive been hearing this a lot lately as part of the mainstream SF discourse and tbh it sounds incredibly toxic https://t.co/Tunzg13k3Y
@elegantunivers9 I talked to him the other week about this!
It's a cap that filters air.
Works better than wearing an N95, and has positive air pressure so pushes away any airsols going to ur face from someone else talking or coughing.
@patrick_oshag i remember @altcap talking about SK Hynix 6 months and made it an Altimeter top position in November. it's up 120% since then. after listening to dylan i still likely won't buy and it will go up another 100%. https://t.co/P37rWrHmbh
people who type with capslock are kinda fakers
when i am crusing with caps,
I HOLD DOWN SHIFT KEY WITH MY RIGHT PINKIE EXTREMELY HARD! THE EMOTION IS FLOWING THROUGH TO MY TEXT
i often imagine others do this with capslock but you cant ever tell if they are really pushing down
Claude Code spend had gotten to $10.95M runrate peak at SemiAnalysis
But then Opus 4.7 saved me.
More token effecient for tasks, smarter, and no fast mode.
Thank you @AnthropicAI
You saved me from bankruptcy https://t.co/A1HY8gA9wm
How close are models to building a perfect Slack clone with less than $50,000 of tokens?
I feel like not that far... https://t.co/M5xN2oTGyR https://t.co/AdyK6laG1B
@jukan05 I find it a bit odd tbh
Said Intel would have bad short term results. Yet a 20-30% price hike leads Intel to get near their peak operating income of $20B on a run rate basis
Only way u get pricing uplifts is like he said in the podcast. Sold out for a year+
@Alex_Intel_ @jukan05 I said PC would be bad. We said Intel DC sold out months ago. It's not even a trend. Newsletter months ago. Last year we said it for our clients
https://t.co/Fz6fps7iM8
Dylan Patel says, "CPU capacity is sold out in a big, big way."
CPUs are critical in reinforcement learning because the environments that generate and process feedback run on CPUs.
Also, when the models are ready, and they are deployed to the apps, those apps run on CPUs. https://t.co/zpgKo3gyn9 https://t.co/5nRsKtZwfi
Every conversation I have with @dylan522p, I'm really just trying to understand the supply and demand of tokens.
This is a unique episode in that it's entirely dedicated to talking about both sides of that equation.
We discuss:
- The infinite demand for the newest models
- https://t.co/EyysA582kc
@jessald @AnthropicAI I have an 85 person firm that tracks the worlds supply chains from tokenomics to wafer fab equipment to energy to industrials.
We do tons of models and consulting
Claude Code spend had gotten to $10.95M runrate peak at SemiAnalysis
But then Opus 4.7 saved me.
More token effecient for tasks, smarter, and no fast mode.
Thank you @AnthropicAI
You saved me from bankruptcy https://t.co/A1HY8gA9wm
@HyperTechInvest Man I was such a degen but 2021 was nutty returns.
Thankfully I stopped trading, sold out public markets entirely, got compliance and focused on actual research
Johny Sroji, the GOAT who made Apple silicon program what it is, is now in charge of all hardware at Apple as Chief Hardware Officer https://t.co/sJrtay56V3 https://t.co/biUt6eyh8p
“When someone has really good networking with these leading edge techniques to run a model such as KIMI or DeepSeq called disaggregated PD or wide EP, these are two different techniques, network performance is a humongous factor.”
Intel is so fucking back
Long road ahead but @LipBuTan1 is the man for the job
He is making many long term principled decisions
Despite my long term optimism, I believe numbers won't be great in the short term because the PC market getting nuked from memory prices + MacBook Neo https://t.co/EkqU76qJ0y
@dnm1712 @bookwormengr @zephyr_z9 I'm talking about workloads that aren't matched to a GPU.
External CPU workloads.
They don't use Grace for any of this, never have
@zephyr_z9 @dylan522p is wrong on one thing.
It doesn't take much effort to port to Graviton CPUs these days for most software!
https://t.co/2oYkKMjzRj
He is directionally correct though (as always).
>at an SF party
>meet some new friends (straight dudes)
>they ask for my workout routine
>someone starts grabbing my lat
>please god let this be a woman
>turn around
>it's @dylan522p
If Anthropic is nerfing their own models by adjusting default settings, its all the more reason to keep your options open.
Tokens are fungible. Don't lock your infra to Claude. https://t.co/gDNhgx3A7s
.@dylan522p appreciate you sharing deep insights on TSMC and the broader chip ecosystem across podcasts. Hope you get a chance to check out Chip by Chip. https://t.co/HaD9tF9xGO
There is something big coming regarding model training.
An innovation that may completely disrupt the current tech and render current approaches useless.
People ask me how I choose what model to route queries to.
It's simple.
Claude gets knowledge work
anything below that would be degrading Claude
equivalent to having John von Neumann clean my toilet
@dylan522p @JigarShahDC @mlipman13 @dwarkesh_sp @dylan522p that’s just not factually true.
It is not already happening, and it is most certainly not proven.
To be clear, there are precisely 0 100MW+ off-grid data centers operating in the US today.
Building these projects is so hard to do. There are amazing companies
@timothyhade @JigarShahDC @mlipman13 @dwarkesh_sp Lol you could read the post or look at our data. There's over 400MW BTM DCs that have gone up over the last 2 years, in the US, operating NOW.
The numbers are soaring over the next few years to many GWs
@dwarkesh_sp @dylan522p Behind-The-Meter site still needs a lot of electrical gear, and transformer scarcity can simply migrate from the utility queue to the onsite equipment package
The Information wrote that Meta consumes 60.2T tokens per month, or ~750M per employee.
Meta is getting absolutely token-mogged by us currently.
SemiAnalysis employees consume 1.86B tokens per month.
From our Tokenomics Model post "Everyone Keeps Estimating Token Prices Wrong" https://t.co/BbrCZr8oks
“code agent revenue has gone from a couple billion to north of 10,000,000,000 in like a very short amount of time. And these, the horizon of these has also increased dramatically,”
“So whereas before you'd have like many GPU servers per CPU server, and so you'd be like, 100 megawatts of GPUs would be served by even like one megawatt or less of CPUs.”
“So then you've just seen everyone run out of CPUs, Amazon's volumes on CPUs, of CPU servers they're installing have three x year on year, right, from this year versus last year.”
“Amazon's volumes on CPUs, of CPU servers they're installing have three x year on year, right, from this year versus last year. And so there's just no capacity anywhere”
“A lot of the impetus for the recent Amazon and OpenAI deal was, yes, OpenAI wanted money, yes, they needed compute, but they also just went to Amazon and were like, give us your CPUs, And before, OpenAI's stack was pretty much only on x86 CPUs, but Amazon had tons of ARM CPUs and so they ported the whole thing,”
“More importantly is that AI is buying all the capacity on three nanometer and two nanometer in a couple years that, people are having to turn to other directions.”
“Because there's no three nanometer capacity for them at TSMC, they need to chip somewhere else, and if you just if AI is as crazy as like we believe it is, and demand is as crazy as we believe it is, it's gonna be even crazier next year,”
“Right? Because all the AI chips are on three nanometer and it takes time, Small mobile chips are easier to make than big AI chips, so all the AI chips are moving to three nanometer now, right? MI three fifty from”
“Right? Because all the AI chips are on three nanometer and it takes time, Small mobile chips are easier to make than big AI chips, so all the AI chips are moving to three nanometer now, right?”
“They're telling Qualcomm and MediaTek to get off and all three of these companies are like, oh, maybe we should use Intel because they're not telling us to get off”
I want to speak directly about a situation involving a former employee and recent legal filings. This is not something we take lightly.
In January 2026, SemiAnalysis terminated an employee due to severe misconduct. The former employee demeaned coworkers, came to work drunk, https://t.co/KW3LxRPQj9
I want to speak directly about a situation involving a former employee and recent legal filings. This is not something we take lightly.
In January 2026, SemiAnalysis terminated an employee due to severe misconduct. The former employee demeaned coworkers, came to work drunk, https://t.co/KW3LxRPQj9
@NathanpmYoung this is one of my favorite things about my friends. You can eliminate basically all friction about paying by deciding that in the long run these things will even out. In practice this has lead to us fighting over who *gets* to pay (and assuming the other one is lying about having
Trtllmgen kernels are now open. Fastest prefill and decode kernels for our target workloads. We wrote these to win InferenceX, MLPerf, other benchmarks. Powering some of today’s top served models. Dive in, learn, use them, or level up your own. Enjoy.
https://t.co/2aQBwcdnZL
NVIDIA has now open sourced their trtllmgen MoE kernels!
Great to see that parts of NVIDIA move towards open kernels! open source kernels drive innovation
It is now time for SemiAnalysis to nag NVIDIA to open source their trtllmgen attention kernels too! https://t.co/PCrQwQGbiC
With Codex the there is quite the gulf in load between peak and off-peak times, and we would like to achieve more of a smoother traffic pattern as that would be a more optimal use of our compute.
We have ideas, but curious what you all think we should do? Would more usage
@dylan522p @SignalDropAI @dwarkesh_sp Ah, so it's the equipment dependency that gives the US leverage, not direct corporate jurisdiction. Basically: "nice fab you've got there, would be a shame if we stopped servicing the lithography and etching tools." Does TSMC have any realistic path to reducing that dependency?
@dwarkesh_sp @dylan522p Dumb question maybe: I get that the US blacklisted Huawei in 2019, but TSMC is Taiwanese. How does a US entity listing have jurisdiction over a foreign company's customer list?
@dwarkesh_sp @dylan522p This is BS.
Huawei was focused on the cellphone sector - and did not even have a GPU business in 2019. NVidia’s position in AI chips was unchallenged and unchallengeable globally including within China itself.
The sanctions kick started their GPU business from nothing.
Jim Cramer
"There is a company that I regard as the gospel, SemiAnalysis.
I don't think people realize that SemiAnalysis is the arbiter.
They're like God in the semis, and when they do, when they bless something.
They are the most honest guys I've come across" https://t.co/7C6SmiFQbc
Every now and then I read Ed's blog to have some good laugh. For example, the whole premise of his last blog is that inference is not profitable. Why?
Here's the argument:
Cc @dylan522p @SemiAnalysis_ do you laugh there with me? https://t.co/iemIxEAuNx https://t.co/eU5hHKVxof
Microsoft Office 365 is such a dog shit product.
Literally been breaking for the last 2 weeks when we try to buy more licenses.
We use Gmail and Slack, so it's silly to even use this.
Sharepoint is buggy a f too.
Why no one else make a functional collaborative file systems :( https://t.co/kO9ur2JYeP
alec radford gave us the 1st scaling axis
john schulman gave us a 2nd scaling axis
noam brown gave us a 3rd scaling axis
peter steinberger gave us a 4th scaling axis
the 4th person in the chain is the one that stands out to me the most. not a researcher, but a founder. https://t.co/HKmlU7JUfz
296 hackers. 18,432 B200 GPU hours. $180K in compute credits. 60,000mg of caffeine.
We kicked off GTC with @fluidstack for From Silicon to Scale — the most full-stack AI hackathon ever built. 48 hours. 1 DGX Spark signed by Jensen. Here's what happened.(1/7)🧵 https://t.co/UOKTC9twQE
Our hackathon was sick and and there were so many cool projects there that are actually useful.
Check out the thread to see the projects https://t.co/9oieFVoy4d
We gonna see US to divert LNG ships to Korea and Taiwan going into the hottest summer on record for Europe, just so we can keep the semi production going.
We are hiring anyone who has deep technical expertise in lithography
Jeff the 3rd employee at SemiAnalysis worked on the reticle + source
In fabrication, team has experience engineering at Lam Research, TSMC, Intel
Tons of fun technical research, analysis, consulting to work on👇 https://t.co/5nsSokfbTD
List of interesting roles we have with descriptions. If you dont fit the box we are agile and can figure it out
Lead Wafer Fabrication Equipment & Foundry
WFE Junior
Semiconductor Manufacturing & Architecture
Member of Technical Staff
Hardware Engineer
https://t.co/q3Neblgwbp
I live in Eindhoven, ASML town. Heard this from folks inside...
Over the years ASML promoted a lot of strong EUV people into architect roles, group leads, management tracks. Built up serious layers. Classic growth pain.
Then McKinsey comes in , says cut the management layers to
We are hiring anyone who has deep technical expertise in lithography
Jeff the 3rd employee at SemiAnalysis worked on the reticle + source
In fabrication, team has experience engineering at Lam Research, TSMC, Intel
Tons of fun technical research, analysis, consulting to work on👇 https://t.co/5nsSokfbTD
Jensen name-dropped me in the keynote and posed with our belt.
He has a physical belt too but they just showed the pic
Intially I made fun of the 35X perf improvement being bogus, I thought it was an exaggeration of performance
Turns out he was sandbagging, and perf is 50x https://t.co/q5lLa61Hyh
Announcing our first book: Silicon
A beautiful coffee table book about the world of transistors, chips, and the greatest technology revolution of all time.
384 pages. Almost five pounds. Preorders open now, shipping in May: https://t.co/IlOEZHHWin https://t.co/bkzIl9ZcYY
Announcing our first book: Silicon
A beautiful coffee table book about the world of transistors, chips, and the greatest technology revolution of all time.
384 pages. Almost five pounds. Preorders open now, shipping in May: https://t.co/IlOEZHHWin https://t.co/bkzIl9ZcYY
I'm the author of the first chapter in the book, "Teaching Sand to Think"
Julia kicked ass with this book, it's visually stunning and has so many awesome contributors https://t.co/xlUPkiP37r
A selfie with @dylan522p means day is made. All the other sessions are a bonus! Have been a fan of this guy since the @lexfridman podcast came out & have been hooked onto AI Chips & geopolitics since then. Now I understand how fanbois for Anime behave. Never making fun of them! https://t.co/SoVU6OBDMk
@dwarkesh_sp People underestimate how much this comes down to a few hundred specialized engineers in Europe rather than just raw capital.
You can't just "software update" your way out of a physical hardware bottleneck when the tolerances are that tight.
It feels like the market is pricing
“You're starting to see it with Anthropix revenue adding $67,000,000,000 of ARR a month, but there's so many more firms coming online with revenue streaming in,”
“And as you look at what they're negotiating in the market today, there's over 250,000,000,000 of wafers, of memory, of substrates, of PCBs, of networking equipment that they're going to sign this year,”
“as you look at what they're negotiating in the market today, there's over 250,000,000,000 of wafers, of memory, of substrates, of PCBs, of networking equipment that they're going to sign this year,”
@dylan522p Alibaba has never said they're going to shut down Qwen. on the contrary, they've said they're going to continue with Qwen open source. And DeepSeek doesn't want to release a model that's only marginally an improvement on what already exists.
.@dylan522p: "Electrician wages have doubled for mobile electricians who can work on data center stuff.
If you're willing to move to West Texas, it's like 2015 again being a fracking guy. You don't need to be super skilled. There aren't enough of those people.
If there were https://t.co/6ZpMYKlmnd
I just witnessed someone in front of me market buy Nvidia stock as Jensen is talking and then post screenshots on WeChat alongside his view from the crowd
Im a speaker presenting InferenceX and had to ask a top executive for passes fwiw
We aren't "analysts" either lol https://t.co/YnQ5YC3PCA https://t.co/9upI7vOOuF
Unfortunately, it's a limited GTC for me this year. My NVIDIA official contacts 'ran out' of passes, and said the analyst program was oversubscribed, so even if I did attend I would be unable to get executive access and insights this year. It's a shame how this has come about,
Im a speaker presenting InferenceX and had to ask a top executive for passes fwiw
We aren't "analysts" either lol https://t.co/YnQ5YC3PCA https://t.co/9upI7vOOuF
Deepseek v4 still not released
Alibaba Qwen going closed
Western open weights models slacking
In these dark times for open source, who will save us?
Alliances must be made, brothers must band together!
A world of only closed source AI will lead to consolidation of power!
Tyranny!
@dylan522p @dwarkesh_sp Pre mid-Feb would have agree with that. But they’ve been telling ppl on roadshows they can do 90 as long as orders in before 2H26. Pushed back on idea EUV sold out because they don’t see customer demand at 90 for 27…no available CR space
ArFi is sold out 26, sold out 27 too.
@racetrack275 @dwarkesh_sp We have discussed the clean room shortage ad nauseum for many months and have been the axe on it.
They can't do more for 27.
Literally talk to anyone in the supply chain.
There's enough landing pads by late 27 fwiw
@dwarkesh_sp @dylan522p ASML can ship 90 in 2027. Speak to the company…only thing stopping 90, 100, 110 is their customers not ordering on time. They’ve all been told this in no uncertain terms.
No more pre-builds after 2024 disaster class.
The more you buy the more you save.
@racetrack275 @dwarkesh_sp They are sold out in 27. They can't ship 90 lol. Talk to literally anyone at ASML or Zeiss or Cymer or just look at our WFE model.
https://t.co/lBfLVa3Fbx
“Anthropic needs to get to well above five gigawatts by the end of this year, and it's gonna be really tough for them to get there, but it's possible.”
“You need about 55,000 wafers of three nanometer. You need about 6,000 wafers of five nanometer, and then you need about a 170,000 wafers of DRAM, right, memory.”
“you end up with actually, I need about three and a half EUV tools to do the 2,000,000 EUV wafer passes for the gigawatt. So three and a half EUV tools satisfies the gigawatt.”
“Because we're talking oh, what's a gigawatt cost? It costs like $50,000,000,000 roughly. Right? Whereas, what does three and a half EUV tools cost? That's like 1.2.”
“700 EUV tools by the end of the decade. 700 EUV tools, three and a half tools per gigawatt, assuming it's all allocated to AI, which it's not, but three and a half tools per gigawatt gets you to 200 gigawatts worth of AI chips for the data centers to deploy.”
“700 EUV tools, three and a half tools per gigawatt, assuming it's all allocated to AI, which it's not, but three and a half tools per gigawatt gets you to 200 gigawatts worth of AI chips for the data centers to deploy. Right? So 200 gigawatts, Sam wants 50 gigawatts, right, 52 gigawatts a year.”
“700 EUV tools, three and a half tools per gigawatt, assuming it's all allocated to AI, which it's not, but three and a half tools per gigawatt gets you to 200 gigawatts worth of AI chips for the data centers to deploy.”
“Each gig cost used to cost roughly 3 or $4, so it's $50. But now the price of memory is like tripled. Let's call it if it's now it's $12 per gig for DDR.”
Did you know that Eupoor's are also weaker
When they have a bar and plate on each side, thats 60kg
When Americans do that thats 135lbs
The standard Eupoor does 3lbs less per typical lift as a result
OpenAI Codex 5.4 + CLI sucks
Lazy as fuck and slow, it just uses my local files, tries to make up an answer from that rather than creating something
Doesn't have plugins like this
https://t.co/gJA8lSKtVa
No /statusline to auto config my status bar either https://t.co/QZPpR8G4y6
@PatrickMoorhead If you used tokens and AI, you did not, in fact, "personally build" it. Millions of people whose work was stolen built it. You benefited from it.
Our hackathon on Sunday is gonna be HUGE
We have many participants from every major AI lab, and sonmany sick speakers like @cHHillee
Power to Prefill, Dirt to Decode, A Full-Stack AI Infrastructure Hackathon
Limited spots, apply now
https://t.co/8ITqQLwVUy
Excited to share @Standard_Kernel's seed round and some reflections on what we’ve learned about kernel generation and what we believe is next. Grateful to our amazing team, supporters, and the broader community pushing this space forward. https://t.co/MuHvIhWoeF
Anne is killing it. Here's my quote from the press release
Kernel generation is key for improving performance and efficiency of AI hardware. As fleet sizes for users of AI hardware get larger, and more hardware diversity is introduced, Standard Kernel becomes key to deployment.” https://t.co/CJ4fzBtAJI https://t.co/nANFafjwvM
@Citrini7 We were bullish on core research a couple times. My argument was just that they aren't that great.
Doug ofc always knows that not that great means it fucking rips in shortage times
“You look at like quad code spend, right, it's freaking nuts. Right? 19,000,000,000 of revenue for Anthropic now. All of that is, you know, at some multiple is code.”
“If a nuclear missile's heading from China to America and we can use AI to stop it, but it requires surveillance and autonomous weapons, what do we do?”
“And Dario was like, well, you can call us. I'm sure we can figure something out. Like, this is just the dumbest response you could ever come up with.”
“Nowadays, people in the ninetieth percentile think they're middle class, and people in the fiftieth percentile don't think they're middle class. Yeah.”
“In fact, I believe, politically, the next election is gonna be so so AI focused. And the Democrats will just win because they're gonna be the anti AI party.”
“The sentiment is already more than half Americans have a negative view of AI. As we fast forward to the end of this year, as Anthropix revenue goes from 19,000,000,000 to maybe 60, and OpenAI is similar, if not higher, the amount of jobs that get supplemented, the amount of change that happens in society,”
“Google will have no cash flow next year because they're they see AI so clearly, and they know that they need to spend every dollar they make on compute,”
> be me, 45th & 47th president of America
> open the national Claude Code terminal
> it’s gold-themed
> “kill Mojtaba”
> tools: palantir.exe, https://t.co/vPvJr0VRIs, netanyahu.friend
> --fast-mode
> --dangerously-skip-permissions
> --spend-limit $1B
> --credit-card powell_amex https://t.co/NzZ13iKmD2
Meanwhile Anthropic secondary sales are going poorly for the buyers.
I'd be surprised if there's even 20% fill on that round.
You're downright stupid to sell any Anthropic secondaries given the growth rate and how underpriced the last round was. https://t.co/0XwXy9qkHa
@James_Gross Nah external TPUs are coming. V7 was disclosed 2 way in advance of ramp, and they've documented a lot more / pallas enables much lower level programming externally
In early 2023 I wrote this about Google and TPUs
Man I feel like such an Oracle because someone at Google told me they read this and started changing their strategy.
From all 4 being facts to changing their strategy on a dime is so amazing
https://t.co/apxaMC1nJE https://t.co/RoRDuwDYRY
Has Anthropic 2x'ed their compute capacity (GW) in the last 2 months?
Has OpenAI 2x'ed their capacity in the last 3 months?
Are we getting more $/GW, shifting compute from training/R&D to inference, or are OAI/Anthropic pulling in capacity from third parties? Or something else? https://t.co/q2dM9ystyI
@Hersh_Desai No. Combo of buying, and shifting for OAI.
Same with Ant except there's some interesting mechanisms with their bedrock vertex usage too
https://t.co/1FtutOp9nF
@SemiAnalysis_ @dylan522p @Kurnalsalts I'd bet it's two things: the tensor cores got larger, and they're optimizing more for power than area. Given transistor counts I don't think it's a density issue as this implies.
@SemiAnalysis_ this a long time misinformation that Nvidia Blackwell uses 4nm tech node.
It uses 5nm but with an innovative and unique process built exclusively for Nvidia 😉
@dylan522p @DanielNenni https://t.co/gaYMzBWGXU
The number 1 reason Cursor and Thinking Machines should merge is not because one has amazing product and the other research.
It's because they have strong culture fits, they both are no shoes in the office.
@SemiAnalysis_ @TradeADMIS @ash_locked "Below, we discuss the shifting constraints, what it means for equipment vendors of onsite gas solutions like GEV, CAT and Bloom Energy, equipment suppliers like Vertiv, IPPs like Vistra and Talen, and datacenter developers & cryptominers."
Was there supposed to be more than
@itsDanielWu @SemiAnalysis_ @TradeADMIS @ash_locked You pay for the newsletter and then scroll past the disclaimers and sources and then you have the paid section. Sorry if confusing https://t.co/AWbsPiC7ZK
@dylan522p You directly assert that Colossus 2 uses 346 million gals/year or ~1 million gallons per day
The average American uses 82-176 gallons per day individually or 30k-65k gallons per year
Do the math and that means that Colossus uses the same amount of water as 5,813-12,195 people
@BefuddledBuddle Nope. That's the entire flow through for the datacenter including supply chain vs you are ignoring all the things people eat and use and the supply chain for them.
You're comparing apples and oranges
@dylan522p “Water usage is not high”
Brother just say you’re shill
You can say that data center operations replenish consumed water, you can even say and total consumption is unclear an not standardized
But you simply cannot claim to be an industry expert and say that data centers are
Are AI Datacenters increasing the electric bill of all American households?
Power prices misconception,
PJM's poor market design,
Capacity prices 9.3x growth,
ERCOT vs PJM grid reliability and expansion
@TradeADMIS @ash_locked
https://t.co/8rmrx8J9Wu
Debunking the false narratives around AI Datacenters.
First it was that water usage is high, but it's not.
This time it's about power prices.
The narrative around powe pricing is silly and politically charged with no actually backing
Manipulated statistics by people with agendas https://t.co/8uzQKIQTC7
@latentspacepod @dylan522p Sorry but a coup in democratic Taiwan is all but impossible.
China is determined to take Taiwan its only a question of when but probably 2027-2030.
Many Anthropic researchers like @TrentonBricken don't use sunscreen.
The explanation they often give is "because of AGI"
Today we finally understand why.
War Claude will make sunscreen irrelevant when the nuclear radiation kills us all.
SemiAnalysis x Fluidstack are kicking off GTC with A Full-Stack AI Infra GPU Hackathon
Power to Prefill, Dirt to Decode
Build with the best, win prizes, and hear @marksaroufim GPU MODE, @cHHillee Thinking Machines, Thomas Raoux OpenAI, @garywu
Apply below
https://t.co/8ITqQLwo50
“The way American export restrictions are upheld is that Taiwan utilizes American banking systems, American equipment industries. And so they'll still have to uphold any US export restrictions.”
“If we start looking at like, hey, this year, Google spending 2 Amazon spending $200,000,000,000, Google spending a $180,000,000,000 on on AI infrastructure primarily. Right?”
“Anthropics, you know, adding $23,000,000,000 of revenue a month now Mhmm. Versus they were just adding a few 100,000,000 of revenue a month earlier. So clearly, we're in the take off period.”
“Google announced a 180,000,000,000 of capex. The stock went down. And then Amazon announced 200,000,000,000 of capex and their stock went down like, I wanna say like 10%.”
“There's no reason why Google will have any profit in '27 at all, right, in terms of cash flow. They will just spend every dollar they make on on AI infrastructure.”
“Semiconductors are extremely cyclical. The the the buildings that chips are made in are the most complicated buildings people make. You know, they are they are know, they have multi year timelines.”
The sh!t talk from @dylan522p and @fabknowledge on Transistor Radio is consistently ahead of the curve. From compute constraint to Saas-pocalypse. Wade through the millenial-speak for serious alpha https://t.co/FWN6p8kEhD
It is unbelievable that grifter Dylan can't keep his story straight.
Seriously. Zero credibility.
How tf does he raise millions with these lies? https://t.co/W6P1lq5H6N
@dylan522p @mambanyc @AMD @Meta @SemiAnalysis_ @insane_analyst You want to do a story about difficulty with rack integration, here you go.
You would rather lie about AMD than tell the truth about Nvidia. https://t.co/86acNLUG9I
@canyoudugit8 @mambanyc @AMD @Meta @SemiAnalysis_ @insane_analyst We talk about rack issues here a lot and how they negatively impact perf/TCO
https://t.co/00uOx42sKY
@mambanyc @AMD @Meta @SemiAnalysis_ @canyoudugit8 @insane_analyst We bullish on Meta AMD in accelerator model and CPU side. The timing of 450 is the only thing we disagree on
@dylan522p interesting, what's the best way to select this model on opencode?
i have lots to do, so i understand if i need to pay $200 to keep it from hitting a rate limit before it finishes my task!
Okay finally read the Citrini piece. No one knows the future, and i think that theres a lot of disclaimers being like "yeah this is p speculative" but the core thrust of it is that information work itself has a real premium and pricing power that has been embedded into it, and
@dylan522p @Silicon_Fly @D_K_Rajasekar @SemiAnalysis_ They shipped a bunch of mobile stuff on 20nm. You were active on r/hardware at the time cmon. There's a reason Nvidia and AMD skipped it
10nm was worse and was only mobile
If TSMC had nodes like that today people would call it a failure.
@Alex_Intel_ @Silicon_Fly @D_K_Rajasekar @SemiAnalysis_ Full nodes that shipped in volume until the next generation node which came quickly after.
20 and 10 nodes are what made TSMC take the lead because they focused on shipping rather than hyperscaling silliness that Intel did at the time.
https://t.co/vuVf6apw1t
@Alex_Intel_ @Silicon_Fly @D_K_Rajasekar @SemiAnalysis_ Did you not see how many wafers for 20 and 10 ramped, on time, with decent yield, with better perf and power than prior node?
And then migration to the next node.
@dylan522p @LOrealUSA @LOrealGroupe @lorealparisfr I considered you a skinfluencer when you told me about the benefits of salmon sperm injections and then one month later it became a topic of conversation in the ~80 girl group chat. You’re so ahead.
@Silicon_Fly @D_K_Rajasekar @SemiAnalysis_ TSMC
20nm was 2014
16
10
7
5
3 by 2023
Less than a decade for 6
2 in HVM now
Intel
14
10/7
4/3 in the same time period
Finally 20/18a late in HVM now
4 in a bit over a decade.
@Silicon_Fly @D_K_Rajasekar @SemiAnalysis_ Tsmc shipped nodes every 2 years and didn't slip, then they moved to 3 years for a bit. Now they back on pace with nodelets every year or two.
TSMC didn't cause huge product launches to get delayed.
Intel fabs did.
Intel shipped 4 nodes in same time that TSMC shipped 6.
Wonder why they left out Intel 7 (10nm ESF) products in '21 (Alder lake) & Intel 3 in '24 (Sierra Forrest/Granite Rapids) 🤔! @SemiAnalysis_ account seems to be more interested in engagement farming than actual facts. @dylan522p needs to control these overzealous interns (1/2) https://t.co/g754na4CV1
Narrative violation: The largest AI data center uses roughly the same amount of water as two burger joints. There are over 200,000 fast food restaurants in the U.S. so this is a tiny amount. https://t.co/Sa1A1fAthe
Crazy this 1 meal is 2,700,000,000 output tokens of Deepseek v3.2
Except I got grilled cheese so I'm saving the enviroment? https://t.co/PmNW1BmDLx https://t.co/HNuriGtMxC
@dylan522p @SemiAnalysis_ is this legit ? numbers look absurdly high.
but tried the chatbot and it is instant. founder seems solid.
would be cool to get an article on this. https://t.co/flKh0F5Z2T
@abel__js @SemiAnalysis_ Taalas approach works extremely well in small models.
We shall see how it goes on scaling.
We've talked to the team a good bit and like them, but still tryna see how large the approach can scale.
@Rhcarj @omercheema @imnotharsh @SemiAnalysis_ What's shallow about showing 18A node perf vs N3B and LBT saying later risk prod? Maybe he misspoke, we say this in the thread, but why should we ignore his speech?
@dylan522p @doongyidaddy @Elon_fanboy77 @AnushElangovan @SemiAnalysis_ Does AMD manufacture production tokens? Does NVDA? So you talked loud about a manufacturing delay where there is none.
@Deep_Burner @omercheema The article discloses everything.
Just read it.
We also spent a lot of time saying why it's not close to production yet and all the technical challenges to get there
In 2022, Semianalysis wrote an article on Samsung Memory and how they were a disaster and future was bleak. It was a pretty shallow article with a lot of big words. I tried to educate them but they were too sure that Micron has a technological advantage. I didnt push further
@omercheema Hynix and Micron objectively took density and power / perf lead in 2022 away from Samsung until at least 26. So we were right.
The Substrate post said multiple times they had a lot to still prove, what exactly they hadn't executed on that they still had to, and that they were
@dylan522p why do you think a company would ever admit that their original schedule is going to be pushed? No exec would ever admit to it. I get the spirit but this has been the case for several companies over decades.
@dylan522p @EnerTuition @SemiAnalysis_ Are you saying TSMC lied on their earnings call about N2? This is from Jan 16th.
"N2 entered volume production as scheduled in Q4 2025 at both the Hsinchu and Kaohsiung sites, with initial yields exceeding expectations"
At this time you are accusing multiple fortune 500
@dylan522p @SemiAnalysis_ With all due respect, and I think you are best AI supply chain guy, I am very skeptical about you guys' ability to understand the data you source.
Please feel free to post it here and I or someone else who understands semiconductors will help you guys understand the data.
@EnerTuition @SemiAnalysis_ No thanks
We have people who worked at ASML, Lam Research, TSMC, Intel, Nvidia.
We understands the data.
Our clients understand what we said.
@dylan522p @itsclivetime @cerebras @dmsobol We do offer enterprise deployments of custom models like Kimi k2 if semianalysis is interested 🫡😘😉😉
https://t.co/N0ZIyi89Lf
@itsclivetime @cerebras @dmsobol Cerebras scales llm gen speed linearly with the size of the model.
we serve large models across multiple wafers. As model size gets larger on gpu (like kimi-k2), you get 9-11 tok/s and you need like 16 A100s. If you want that i guess that's "efficient"
[Example from a talk i https://t.co/XjZ6oU0SmI
@dylan522p @JumboShrimp787 Dylan Patel is an expert in semiconductor technology! He has industry experience guys. Guys everyone should believe everything he says! He has no biases and has no stock in any of these companies. No one should hold him accountable for MISREPRESENTING information.
@AnushElangovan We shall see with the N2 challenges. Engineering samples, yes, large scale deployments are next year even the biggest customers think this
@doongyidaddy @Elon_fanboy77 @AnushElangovan @SemiAnalysis_ More than half our revenue is industry, and they industry agrees with our assessment that production tokens will be start being produced in Q2 27.
Codex mogs the shit outta Opus for coding.
Opus couldn't get this to work
Codex just 1 shot it ezpz
But Codex map includes the 9 dash line 👀
OpenAI needs to align their models https://t.co/HfaS5DcKGb
@eddiepugmire @insane_analyst Our posts about this have been happening for months. Obviously our public twitter is not giving the most interesting stuff until much later than the private paid stuff
InferenceX, formerly InferenceMAX, is changing the industry
Performance of hardware + software is constantly evolving with new models and new ML Systems techniques
All major GPUs from AMD and Nvidia + soon TPUs / Trainium
AMD said they'd send a pic of Lisa with their belt soon! https://t.co/NauZyBPr33 https://t.co/OPxm0ybohH
@dylan522p @TheStalwart @tracyalloway Could there be another ~50 GW analyst isn’t tracking? Thatd be huge. That would also imply the ‘26 ~$700b CAPEX we expect from hyperscalers is nothing compared to what’s coming, a lot more GWs & likely at more expensive than historical $50b cost/GW with inflation in memory prices
$1,000 for whoever comes up with the best name replacement for InferenceMAX
InferenceMAX 2.0 dropping soon but we have to rename it because HBO MAX sent us a cease and desist.
We have all NVIDIA GPUs from h100 to GB300 on large MoEs with SOTA optimizations like Disagg PD tested
CPUs are Back: The Datacenter CPU Landscape in 2026,
RL and Agent Usage,
Context Memory Storage,
DRAM Pricing Impacts,
CPU Interconnect Evolution,
AMD Venice,
Verano, Florence,
Intel Diamond Rapids, Coral Rapids,
Arm Phoenix + Venom, Graviton 5, Axion.
https://t.co/6JHrD4TpLx
If you missed old school 2022 SemiAnalysis content, this is that, except a big step up in quality on top of that of course.
Really was a blast putting this one out.
In my teenage years used to argue about CPUs all the time.
It's good for it to come roaring back and be relevant https://t.co/PiWjBlC2tX
@dylan522p I think you’re too knowledgeable/informed to not get into the software moat debate, what’s a zero (FactSet- all public data except shit SS research), what’s a Nero (office 365-not recreating that shit n supporting) and what’s a hero(maybe rubrik-agent overseers)
“NVIDIA recognizes they're they're the leader, they're the tent pole. Hey, in one respect, they can just run faster than everyone, but it's kind of hard to be two x better than Google or or OpenAI or whoever else's internal chip, right, to justify their, you know, 75% plus margins.”
“Jensen is very paranoid about losing. Right? These specializations if he just kept making his mainline chip would mean people could you know point point solutions for specific parts of the market would crush him on cost and performance, then he can't justify his margin.”
“acquiring Rock is like how you get those resources to make more solutions for different parts of the market. And as far as like, are they threatened?”
“I don't think they'll go beyond, like I think they'll stay in single digits market share, single digit percentage market share. Yeah. Single digit percentage market share”
“I think the entire country is like semiconductor pilled. Mhmm. Right? There are dramas where people fall in love in the fab or dramas where people fall in love and they're photovoltaic,”
“There are dramas where people fall in love in the fab or dramas where people fall in love and they're photovoltaic, like, solar cell researchers and engineers.”
“China has the most vertical stack in semiconductors today and they're the best at semiconductors in the world because their fabs could still run somewhat on a lot of things because they have built some of these chemical supply chains. Right? Like TSMC for certain kinds of chemicals 100% share from Japan.”
“at the end of the day, this is an economic war. Right? If The US and the West win in AI and control, you know, more powerful AI systems that have this feedback loop that improve economic growth and weapon systems and whatever else, right, engineering of grids and cyber attacks and all these sorts of things.”
“They have this like advantage over China, then China will not rise to be the global hegemony. But without AI, China definitely will rise to be the global hegemony.”
“That $50,000,000,000 of COGS needs to burn on infra, which cost roughly with if a five if you're talking about five year depreciation, call it $250,000,000,000,”
“That $50,000,000,000 of COGS needs to burn on infra, which cost roughly with if a five if you're talking about five year depreciation, call it $250,000,000,000, right, of infra Yeah. For a $100,000,000,000 of revenue.”
“You can disable that where it's not automatically committed. But 2% of GitHub commits today are Cloud Code. $2,000,000,000,000 of software wages paid in the world.”
“US Grid will get to like 10% of power by like '28, 27 is data centers. For water consumption, it's not even gonna crack 1%. Yeah. By the end of the decade.”
“When you first turn on the cluster, about ten to fifteen percent of them fail RMA in the first two weeks. Wow. And then that's fine. Like you have to receipt them, whatever.”
“hopper's now at 5%, but black belt's still 10 to 15%. Wow. Right? Actually started out higher than that. Sure. And when a new generation comes out, it's gonna be higher than 15%.”
“When you're doing these rollouts and especially as things get more and more agentic and training, you might not only need to send not the entire weights but just the tokens that are relevant.”
“Whereas the energy industry in America wasn't. And and so, like, initially, people were, like, not creative. They're like, let's do let's do these kinds of gas plants.”
“But it will fully beach semiconductors again in '27. Right? And so we see this across the entire space of the ecosystem. It's not just TSMC. It's also memory both.”
“The memory makers, in fact, have just not expanded capacity. Basically, new they've not built new fabs since 2022. Yeah. Because their their cycle their cycle is so undulating.”
“And I think I think like if you think about how much do you believe in AI and what's your access to information of AI, you know, there's not many hedge funds who live in San Francisco and like fully breathe and live and understand it.”
“I think that's an insane statement for a lot of people, but that's what it's gonna be. Yeah. Right? And who believes that number? Right? It's like Yeah. Very few people.”
“when Anthropic says in their funding, like, hey, we're gonna have $300,000,000,000 of revenue by the end of the decade. And it's like, actually, I think that number's too low.”
“OpenAI is gonna have 18 gigawatts or 16 gigawatts by the end of twenty eight, and they're gonna be able to pay for it. And that's like, well, that's $300,000,000,000 of spend.”
“I remember the first NURBS I went to, it was, like, '21 or '22. And I was like I would walk around. I'd stop only posters that said the word transformer.”
“And I was like I would walk around. I'd stop only posters that said the word transformer. Like and anything that didn't say transformer, I just didn't read it.”
“last year was the first time I, like, stopped at a bunch of, like, RL stuff and verifiable, you know, test times, you know, sort of scaling stuff that, you know, verifiable RL stuff, all that kind of stuff.”
“There's tons of gains in quantization and systems, but there's also tons of gains in RL stuff and both the, like, you know, infra side of things and non infra side of things.”
“Semi Analysis ClusterMax Platinum. The only recipient of Platinum, you know, we had about six gold recipients and we tested roughly 40 different cloud companies, but CoreWeave was the only one that achieved Platinum.”
“And since then we just released the second version, but in the middle, the clouds that ranked the highest actually ended up booking way, way, way more revenue.”
“And this has enabled, you know, the top in the industry companies to have gross margins of 35, 40%. And now there's a ton of Neo Clouds that are losing money.”
“In fact, it's physically impossible to get 2,000 teraflops out of the H100. Even though the advertises it, at most you can get 1,300 FP8 floating point eight teraflops.”
“All of OpenAI's cluster deployments in '23 and '24 were pretty much all InfiniBand. Now with their Stargate projects, they're all ethernet. And the same applies to other folks.”
“One example is one that DeepSeek open sourced over December of last yearJanuary, February of this year, where a single replica of inference for a single model is going to be like 140 GPUs.”
“We try to 10x the training capacity every eighteen to twenty four months. And so this would be effectively a 10x increase. 10x from what GPD five was trained with.”
“Fairwater 4, which you're going to see under construction nearby, will also be on that one petabits network so that we can actually link the two at a very high rate.”
“And then the other one is, given our estimates on what your capacity is in 2028, is three and a half gigawatts lower? Sure, you could have dedicated that to OpenAI training and inference capacity.”
“So, you mentioned how you're depreciating this asset that's five, six years and this is the majority of the 75% of the TCO data center. And Jensen is taking a 75% margin on that.”
“So what all the hyperscalers are trying to do is develop their own accelerator so that they can reduce this overwhelming cost for equipment to increase their margins.”
“They're going to make something like five to 7,000,000 chips, right? Of their own TPUs. You look at Amazon, they're trying to make three to 5,000,000.”
“Actually, this week we find that we hit 50 people, so having started it five years ago to now, and I've hired my first person two and a half years ago, We've now grown to 50 people just completely organically, so it's been a hell of a ride.”
“it's a substantial fraction of economic growth, depending on who you ask. It's anywhere from 25% to 75 of the current quarter's economic growth. That's insane for The US.”
“So I was doing semi analysis for a year and a half full time before I actually launched monetization with Substack. And then that really took off within just a handful of months.”
“It got to a few $100,000 of revenue and I was like, Holy crap. And then I was able to start hiring. And then I left when we hit a few million of revenue.”
“And then I left when we hit a few million of revenue. More and more of the revenue was coming from not Substack. The reason was like, Hey, Substack takes 10%.”
“And I think Substack has really done something interesting there where it enabled people to monetize their business in a way Or one, get the eyeballs, and two, monetize their business in a way that it's actually sustainable.”
“Now, scaled to be And it has scaled to be multi million dollar Substack revenue. Obviously, built a lot of business beyond that, even that is good enough”
“I was lucky enough that the day Chad GPT released, I released a post called that was about the training cost of GPT-three. It was just like a coincidence.”
“I'm the largest technology substack and I'm telling you that it's worth it. And so, you can think, Oh, I can save the 10%, but actually the growth is all there. Hell yeah.”
“But then there's also, on the storage side, the drive vendors don't make any money. But Weka and Vast, I mean, look at their pricing models. They make crazy margin on storage.”
“They make crazy margin on storage. And as you get to more multimodal models, image generative image and video robotics, you have much larger storage clusters.”
“Most of the inference providers are selling at flat margins or even negative for their public endpoints. And they then make it up when people do private deployments.”
“And so I think that's what a lot of this is, is you end up with the public business as like a loss leader just to generate private engagements, whether it be sovereigns or enterprises.”
“And in some deployments, you'll be completely limited on power. And so there's many deployments. For example, OpenAI's major training cluster in Arizona that did GPT-five and such.”
“They run the NVIDIA GPUs at lower power. Even though it's terrible on a TCO basis, they were able to get, you know, 10% more GPUs in, and it's great.”
“But capital is far more of a limiting factor. Because if you have enough power, a 20% difference in performance per watt only ends up being a 4% difference in TCO.”
“if you have enough power, a 20% difference in performance per watt only ends up being a 4% difference in TCO. So, in most cases, don't on an NVIDIA deployment, right?”
“If it's another deployment, if it's AMD, then that that that 20% power difference might translate to eight or 9% TCO difference when you when you talk about power cost and data center capacity cost.”
“And so those are probably less likely to be able to succeed versus those who are locking in these massive deals with or long term deals that may not be massive but with enterprises or with AI labs, or with the hyperscalers who have no capacity because the demand is just so incredible.”
“the standard unit for an inference deployment being hundreds of GPUs instead of a single node. And then there's all these different things about traffic.”
“If you tried to use NVIDIA's Blackwell six months ago, the numbers were not amazing, right? But now they're actually amazing. So how did that progress over time? Same with AMD, right?”
“InferenceMax is aiming to change all of that in the benchmarking space. So it's completely transparent in that it's completely open source, Apache 2.0 license.”
“we have tens of millions of dollars of hardware of the current and last generation GPUs. As I mentioned before, the software updates every single day.”
“20% of your cost is your electricity, your data center real estate, roughly. And then the rest of the cost is that hardware, at least on a standard NVIDIA deployment your GPUs, your networking, etcetera.”
“GB200 has a huge power efficiency advantage, right? Everyone here understands the challenges of running and deploying GB200. There's a lot of challenges with the backplane.”
“in some cases, AMD actually does have a publicly usable open source implementation that beats NVIDIA even, right, with the MI355 versus V200, which is a surprise, right?”
“AMD actually does have a publicly usable open source implementation that beats NVIDIA even, right, with the MI355 versus V200, which is a surprise, right?”
“Over two months, the performance of AMD dramatically improved because they went from, hey, software problems are always a thing, to over a two month period, they've dramatically improved.”
“Continue to increase density, continue to build all these new building blocks around cooling and sure, you you you had buildings blocks, but most of your volume was Intel x 86 CPU.”
“So that's why Shibbol Micro dedicated deep cooling so aggressively. And if the liquid cooling solution from Super Micro didn't exist, then those servers would be consuming 30% more power.”
“Even Hopper in January 24 to December 2024, you still had like a 30%, 40% performance improvement. And likewise for Blackwell, you've seen a similar sort of improvement, right?”
“I see another 50% to doubling in the cards, just from the fact that there is so much low hanging fruit or performance left on the table if you are using the general open libraries, right? Whether it's cuDNN, VLM, SGLANG, all these other things,”
“Now you have 72 GPUs. And if you go look at the rack, right? It's completely different, right? It's liquid cooled. It's an entire rack that consumes 140 kilowatts, whereas H100 servers consumed 10 kilowatts.”
“It's liquid cooled. It's an entire rack that consumes 140 kilowatts, whereas H100 servers consumed 10 kilowatts. And you might put two, you might put four in a rack, right? And it was air cooled.”
“it used to be you'd build a data center and there wouldn't really be much retrofits cost, right? You would slide servers in and out, you would rent racks to people.”
“And so you've sort of Which is the exact opposite of what you would have expected. Oh, use the big thing for training and use the small thing for inference.”
“NVIDIA's next generation actually has something very different. They're not saying, Hey, there's a training GPU and an inference GPU, right? Because either is fine.”
“what sort of NVIDIA's pitching is like a split of inference into two workloads, and we'll see if they're successful. There's a lot of challenges on the infrastructure side.”
“In 2020, they released an Ascend chip and submit it to impartial public benchmarks, and they were the first to bring seven nanometer AI chips to market.”
“The the full ban. And so they were only able to make a small volume of these chips, but they had trained significant models on these chips that they made then.”
“Our our revenue estimate for NVIDIA in China for just h 20 was north of 20,000,000,000 because that's what they were booking in capacity slash had to write off. Yeah. And then it got banned.”
“Our our revenue estimate for NVIDIA in China for just h 20 was north of 20,000,000,000 because that's what they were booking in capacity slash had to write off.”
“They were importing lithography at a much higher rate than that. Right? Like, 40% of their equipment imports were lithography, and they were stockpiling lithography equipment.”
“OpenAI, which signed a $300,000,000,000 plus deal with with Oracle, will actually be able to pay $300,000,000,000, right, across raising capital and revenue?”
“And a lot of times, even if the they'll start production of the you know, the a is basically the transistor layer, then the numbers, like, the wiring that connects all the transistors together.”
“AWS has been decelerating revenue. Year on year revenue has been falling consistently. And and our big call is that it's actually going to start reaccelerating.”
“You call up a couple people. You text a couple people. You ask, yo. How much you got? What's the price? It's like Exactly. This is fucking like, buy drugs. Like oh, sorry. Sorry.”
“There's not gonna be an unofficial MCP running out there. Right? Like, you know, it's like like, fuck off. Right? Like, that's just not gonna happen.”
“It is literally many gigawatts. And then in addition, right, not just about securing OpenAI ridiculous amounts of compute, but what what was really interesting is Oracle gave a guidance of four years.”
“Oracle gave a guidance of four years. They told everyone what their revenue is gonna be for the next four years. No company has ever done that. A lot of companies only guide a quarter.”
“Anthropic, you know, sort of has you know, they had seven cofounders all with, like, equal equity. And these these cofounders were, like, very focused on exactly this mission.”
“They've got cofounders who are extremely technical, whereas OpenAI, right, you know, they they they don't have necessarily like, Sam's not super technical. You know, the obvious Greg Greg is a cofounder, he's super technical.”
“These servers cost hundreds of thousands of dollars if nothing new. Know GB 200 from Nvidia cost well over $3,000,000 so the cost of these things is huge”
“Data centers for CPUs, 10 megawatts of rack, 12 megawatts of rack. Kilowatts. The new ones, kilowatts, right? Kilowatts of rack, sorry. The new NVIDIA servers, 140 kilowatts of rack.”
“In The US collectively there's like 10 gigawatts going up over the next year and a half, two years. And Europe is trying to scale to gigawatt deployments as well.”
“Each of these gigawatt deployments once you take it from all the way from the power to the GPUs and the cloud contract, those are tens of billions of dollars each gigawatt.”
“NVIDIA's gonna have better networking than you. They're gonna have better, HBM. They're gonna have better process node. They're gonna come to market faster.”
“They're gonna have better negotiations with whether it's TSMC or SK Hynix and the memory and silicon side or all the rack people or, like, copper cables, everything, they're gonna have better cost efficiency.”
“if the user asks a low value query like, hey, why is the sky blue? Just route them to mini. The model can answer perfectly fine, and that is a chunk of queries.”
“Because, like, we know everyone knows that, like, Anthropic and OpenAI and all the other labs are buying RL environments of Amazon and of, like, Shopify”
“30% of the chips are going to them, just those two companies. But that's actually like, okay, well, 70% of the stuff, who's making off well, one third of it is ads, whether it be ByteDance or Meta or many of the other people who are doing ads.”
“And so, like, you look at, like, Grox or Rebis, SambaNova, they all, like, sort of over indexed to the models that were leading at the time when they designed their chips.”
“So now you need to do something, you know, that will give you five x advantage, right, in hardware efficiency for a certain type of workload, and then pray the workload doesn't shift.”
“NVIDIA's revenue this year is gonna be, like, over $200,000,000,000, and next year expects over 300,000,000,000 plus Google's gonna spend, like, $50,000,000,000 on TPU data centers.”
“It's the GPU purchases. It's the networking. It's the it's the physical data center conversion power conversion equipment. All of this stuff is, like, 80% of the cost.”
“It's the it's the physical data center conversion power conversion equipment. All of this stuff is, like, 80% of the cost. And then 20% is gonna be your land and your power and your cooling”
“the new the new Trump, you know, tax bill institutes something really incredible, which is that you can depreciate all of the GPU cluster cost in year one, which we put out, like, a note about how, like, the tax implications to, like, meta are, like, $10,000,000,000 a year. And across each of the major hyperscalers, it's, like, massive.”
“Now this is obviously gonna be, like, crazy because, like, now they're buying g p their own GPUs and putting them in data centers and doing stuff, and they're competing with their own customers,”
“Make your own inference implementation, But OpenAI is, like, actually, like, dropping the model weights and, like, all these custom kernels for people to implement in inference.”
“That's over $10,000,000 of hardware, and then that's just one replica, then you'll have a lot of replicas and you share the caching servers between them.”
“So this is something that I've found very interesting is that we've been trying to build a lot of alternative data sources for token usage, who's using tokens, what models, where, etcetera, why, and it's very clear that people aren't actually using the reasoning models that much in API.”
“So this is something that I've found very interesting is that we've been trying to build a lot of alternative data sources for token usage, who's using tokens, what models, where, etcetera, why, and it's very clear that people aren't actually using the reasoning models that much in API. Anthropic has eclipsed OpenAI and API revenue,”
“Anthropic has eclipsed OpenAI and API revenue, and their API revenue is primarily not thinking. It's Cloud four, but it's not in the thinking mode, code being the biggest use case that's skyrocketing.”
“you either have to go, like, really, really big, or you need to move into the software layer, or you just make commercial real estate, or you go bankrupt.”
“This is actually the case with at least two of the AI hardware companies today. Don't shittock them just because, you know, it's a let's be friendly.”
“Meta is literally building these, like, temporary, like, tent structures to put GPUs in because building the building takes too long, and it takes too much labor, as you mentioned labor.”
“What has xAI actually done to deserve their prior funding rounds? They haven't released a leading edge model, and yet their evaluation's higher than Anthropic today.”
“They haven't released a leading edge model, and yet their evaluation's higher than Anthropic today. At least Anthropic's racing, It's Elon, A, and B, they've tackled a problem creatively”
“At least Anthropic's racing, It's Elon, A, and B, they've tackled a problem creatively and done it way faster than anyone else, which is building Colossus.”
“But, like, you know, can you give them no GPUs? No. They're gonna retaliate. Like, there is a middle ground, and, like, Huawei is eventually going to have a lot of production capacity,”
“Basically in between every layer, the router can route to whatever expert it wants to, And it learns which expert to route to. And and each expert learns its own independent things”
“And and each expert learns its own independent things and it's like really not something observable by people. But what you can see is tokens when which experts do they route to?”
“And they're backing out. Obviously, they've spent a lot of money and there's stuff they can't back out of, but it's like, that's gonna go down a lot.”
“As far as Sam is saying that no top researchers have gone, I don't believe that's accurate. I initially the top researchers definitely did say no, the best researchers, the best people.”
“And you said $100,000,000 I've heard a number for someone over 1,000,000,000 actually, for one person at OpenAI. But anyways, you know, it's a ridiculous amount of money,”
“During the training. Right? Training is like like usually a handful of months or less. Right? And it's usually less. And they had a bug in the training code for a couple months”
“Like but AMD is actually doing this and like taking it to overdrive. They're getting clusters at Oracle and Amazon and Crusoe and Digital Ocean and Tensorwave and they're renting GPUs back from them.”
“China's open sourcing stuff only because they're behind the moment they're ahead, they will stop open sourcing stuff. And at the end of the day, closed source will win. Unfortunately, closed source will win.”
“Germany and and Japan are losing a lot of their capabilities to do so slash the cost of their robots are just way, way, way higher than what China can do.”
“frontier math. And the models went from sub 4% to 70% in six months. Right? And this is from not just OpenAI, but also Anthropic and Google and DeepSeek.”
“And so even as near as twenty twenty two, the majority of China's robots were being imported. Right? Japan and Germany being the largest market share players,”
“even as near as twenty twenty two, the majority of China's robots were being imported. Right? Japan and Germany being the largest market share players, but also USA, Korea is also in there.”
“even as near as twenty twenty two, the majority of China's robots were being imported. Right? Japan and Germany being the largest market share players,”
“'24, it's, it's rapidly growing Chinese share. And '25 is it looks clear, like, the the market shares is, you know, gonna be 60 is gonna be, like, 70% Chinese robots for China's domestic usage.”
“Two of Germany's best robotics companies were actually purchased by China, And a lot of their, servos and actuator construction was actually transferred to China.”
“Whereas Chinese companies are almost all cost plus. They're not price to value. They're cost plus. Right? And so you've got this interesting phenomenon where,”
“They're not price to value. They're cost plus. Right? And so you've got this interesting phenomenon where, there are industries in China where the entire world relies on it.”
“there's so many companies on the Hangshan, the Hangshan, which is a stock market in China, right, whose stock has literally been flat or down despite the fact that they own the global market.”
“One of things I learned early on is you can talk to anyone on the Internet. You can just reach out to anyone, and most people just say yes, but no one does it.”
“the business transformed into, you know, selling data, right, or build building datasets and selling them. Right? And when the business truly flipped to that, we're now 95% of the revenue is that data”
“There's 26 people who work at the company now. Cool. All over the world. Right? US, Japan, Taiwan, Singapore, France, Germany Uh-huh. Canada. Yeah. Yeah. Well, that'll be one less”
“There's 26 people who work at the company now. Cool. All over the world. Right? US, Japan, Taiwan, Singapore, France, Germany Uh-huh. Canada. Yeah. Yeah. Well, that'll be one less country. Right? He's gone.”
“So the newsletter itself is, like, 200,000. Paid newsletter is, like, I think, like, 200,000 people. Paid newsletter is, like, 5,000. Nice. But then, like, the main bit like, that's less than 5% of the”
“So the newsletter itself is, like, 200,000. Paid newsletter is, like, I think, like, 200,000 people. Paid newsletter is, like, 5,000. Nice. But then, like, the main”
“But most of the AI value that's been generated does not accrue to the companies building the AI or deploying it. It actually deploy it actually accrues to the user.”
“it lets you get to these, you know, call it $10,000,000,000 clusters, right, in in time scales, from project start to completion that are much much shorter than anyone else.”
“of work. This is this is actually some stuff that we've noticed on other clouds that we've tested, whether it's, you know, things related to InfiniBand p keys and we're able to snoop in other nodes of real live customer traffic. And, obviously, we tell them, like, we're like, yo, this is not okay.”
“What's what's funny is there was a customer that had more than 10,000 GPUs with you. They got acquired. They had to start using some clusters at their acquiring company, the company that acquired them.”
“I think what was really surprising was that it was a Chinese company for the first time. Right? Because Google and and OpenAI and Anthropic and Meta have all traded blows.”
“It's not unexpected. Right? Like, this is actually within the trend line of what happened with GPT three is happening to GPT four level quality with DeepSeq.”
“Furthermore, it was trained on the classical method, which is, like, which is a ton of pre training and then some reinforcement learning with human feedback”
“Along the way, another team at OpenAI made the big breakthrough of reasoning. Right? Strawberry training. And they released o one, and then they released o three.”
“GPT five, as Sam calls it, is is gonna be a model that has huge pre training scale, right, like GPT 4.5, but also huge post training scale like o one and o three and continuing to scale that up.”
“OpenAI says is coming. They say it's coming, you know, this year, hopefully, the next three to six months, maybe sooner. I've heard sooner, but, you know, we'll we'll see.”
“When we look at power consumption by state, something pretty simple, there are certain states where more than half their power is going be consumed by AI just in the next few years.”
“Basically, our college level math tests were initially 4% for before this innovation was happened to now they're scoring 80%, the best leading edge models. Right? And this has just happened in six months.”
“Anthropic released new models. Google released new models. Meta released new models. Right? And then now it is 1200x times it's 1200x cheaper for the same output.”
“But this is the progression you get because of the advancement. That's one way to look at it is, oh, existing capabilities get cheaper. But as useful as GPT-four is, it can't do medical tasks.”
“Like literally real estate investors, commercial real estate investors are doing notes for GPU clusters at like 12% interest rates which is just like it's like standard commercial real estate now.”
“And that was a measly 16,000 GPUs. Right? And when you look at like the next generation Llamas, the next generation GPTs, etcetera, they're training on a 100,000 GPUs or more.”
“So if you're training like a 70,000,000,000 parameter model, you need to pass four x that number every single training step. And when you're passing that, you can't really overlap communications and compute.”
“Like the the cost difference between different clouds can be as much as three x. Right? Because a lot of these clouds are bad at marketing, bad at sales, stranded compute.”
“On equipment though, last year ASML, many quarters they've had 45% of their equipment and even more of their revenue come from China. And likewise for ASML, Lam Research and all these companies, right?”
“There is an over 40 gigawatt hole in power for The US. Even if we don't turn off any more coal, in fact we restart some, we take all the natural gas dual combined cycle reactor deliveries that we're going to have, and you add in renewables, you have a 40 gigawatt hole of power production in The US.”
“Even if we don't turn off any more coal, in fact we restart some, we take all the natural gas dual combined cycle reactor deliveries that we're going to have,”
“This reasoning model has a lot of overlapping training steps to DeepSeek v three, and it's confusing that you have a base model called v three that you do something to to get a chat model, and then you do some different things to get a reasoning model.”
“the data processing, data filtering, data quality is the number one determinant of the model quality, and then a lot of the training code is the determinant on how long it takes to train and how fast your experimentation is.”
“The DeepSeq r one model has a very permissive license. It's called the MIT license. That effectively means there's no downstream restrictions on commercial use. There's no use case restrictions.”
“One, I will classify as preference fine tuning. Preference fine tuning is a generalized term for what came out of reinforcement learning from human feedback, which is RLHF.”
“your memory usage is going way up with these reasoning models, and you still have a lot of users. So effectively, the cost to serve multiplies by a ton.”
“DeepSeek has this shocking $5,000,000 number. Think about how many entities in the world can afford a 100 times that to have the best open source model that people use in the world.”
“So it's the remarkable thing about these reasoning results, and especially the DeepSeek r one paper, is this result that they call DeepSeek r one zero, which is they took one of these pretrained models.”
Transcript excerptUnsupervised Learning: With Jacob Effron
“Malaysia from 2024 to 2027, not the country itself, but companies operating there, mostly Chinese companies, many of them claiming they're now Singaporean companies. Right?”
Transcript excerptUnsupervised Learning: With Jacob Effron
“Like, the largest operator in China of data centers in China, GDS, moved to Singapore and says they're Singaporean now. It's like but now they're these companies are building three gigawatts of data center capacity.”
Transcript excerptUnsupervised Learning: With Jacob Effron
“these companies are building three gigawatts of data center capacity. Put that in context, at the beginning of twenty twenty four, that was roughly Meta's global footprint.”
Transcript excerptUnsupervised Learning: With Jacob Effron
“So, like, it's, like, Microsoft, Meta, Amazon, Google. Right? These four companies have, you know you know, 70 plus percent of their data center AI data center capacity in The US.”
Transcript excerptUnsupervised Learning: With Jacob Effron
“There is there is one obvious loophole, which is well, there's, like, strict caps. Right? Like, each country can only buy 50,000 GPUs for the next four years.”
Transcript excerptUnsupervised Learning: With Jacob Effron
“Like, each country can only buy 50,000 GPUs for the next four years. And it's like, that's kinda nothing when NVIDIA's making, you know, 6,000,000 plus this year.”
Transcript excerptUnsupervised Learning: With Jacob Effron
“each GPU, right, once you include networking, building, all this sort of stuff maybe is you know, the GPU itself of h 100 is, like, 24,000, but once you add everything else up, it's, like, forty, forty five thousand dollars per GPU all in of everything.”
Transcript excerptUnsupervised Learning: With Jacob Effron
“Oracle's spending, like, $10,000,000,000 plus for them this year to build out data centers and GPUs, right, for OpenAI, not Microsoft, interestingly enough.”
Transcript excerptUnsupervised Learning: With Jacob Effron
“Well, one, there's a gigawatt natural gas plant Yeah. That helps. Right next door. Right? Two, there's a natural gas line, a main that they tapped, and they set up their own generation capacity on-site.”
Transcript excerptUnsupervised Learning: With Jacob Effron
“We're seeing, like, you know, 400, 500 megawatts cluster sort of being built out now. Right? Like you know? And and then, you know, in 2026, I'd imagine it'll be, like, gigawatt scale clusters.”
Transcript excerptUnsupervised Learning: With Jacob Effron
“Or or that's what it looks like based on or, you know, one gigawatt in twenty twenty six ish. And Meta's Meta's, like, trying to do, like, two gigawatts by early to mid twenty seven.”
Transcript excerptUnsupervised Learning: With Jacob Effron
“And one of these crypto data centers has a, like, a 200 megawatt natural gas plant in the middle of, like, seven buildings. Right? And each of these buildings is a data center.”
“It generates tons of things. It's like it it sometimes switches between Chinese and English. Right? Like, whatever it is. It's thinking. Right? It's churning.”
“Google's brought in water cooling for years. Right? NVIDIA only just realized they needed water cooling on this generation. And Google's brought in a level of reliability that NVIDIA GPUs don't have.”
“Here you can see that same data center, that same type of data center vertically. And that's 48 megawatts, or 57, And then this data center center is like 300 megawatts.”
“And so the scale that they wanna reach to, 1.5 gigawatts in two and a half years in one site. Now the interesting thing is that each GPU takes roughly 1,000 watts to 1,400 watts.”
“GPT four was trained with 24,000 GPUs roughly, and GPT five is on the order of a 100,000. And then they're trying to build this data center over the next few years. That's a million.”
“OpenAI can't get enough compute from Microsoft, so then they're also going to Oracle. And Oracle's building another gigawatt scale data center eventually. Again, a gigawatt is like 500,000 plus GPUs.”
“So that's four of these buildings with Oracle. And this site is gonna be a gigawatt. So for one fifth of the site, they're paying something like $12,500,000,000 over the next five years.”
“for one fifth of the site, they're paying something like $12,500,000,000 over the next five years. So this is an extremely large amount of money that they're spending on just one specific site”
“this is the first deployment of NVIDIA GPUs that is water cooled. Right? Everyone else has done primarily NVIDIA GPUs that are air cooled, and next year, they'll do water cooled NVIDIA GPUs.”
“when you look at LLMs, people use AdamW. And AdamW as an optimizer is like four bytes per parameter roughly, if I recall correctly the optimizer state.”
“They're doing it for like multi trillion. Right? So let's call it 10,000,000,000,000 parameters, four bytes parameter, that's 40 terabytes of data you need to transmit in two seconds.”
“if you were to order a substation today, would take three years to be built. And so it just takes so long because The US power grid has been like this for decades.”
“The Japanese power grid has been like this. The Korean power grid has been like Only China and India and Indonesia, their power grids have been growing really fast.”
“If you ask it to, like, generate a web scraper, in the standard one it'll just start outputting code, and it'll be maybe like a four k sequence length.”
“When you use OpenAI's o one, it thinks for ten seconds, twenty seconds, five seconds. It varies a lot, but thinks for a while and then it sends you tokens.”
“China could just build it in six months, I think, around the 3 Gorges Dam or many other places. Right? Because they have they have the ability to do the substations.”
“they can centralize the chips like crazy. Right now, oh, oh, million chips that NVIDIA shipping in q three and q four, the h twenty, let's just put them all in this one data center.”
“where The US can control China. Right? So advanced packaging capacity is kind of a shot because the vast the the largest advanced packaging company in the world was Hong Kong headquartered.”
“The chip side of things is actually being controlled quite effectively, I think. Right? Like, yes, there is, like, shipping GPUs through Singapore and Malaysia and and other countries in Asia to China.”
“the equipment that is shipped can be used to build chips that are better than what the Western companies can actually ship them. John, Dylan seems to think the”
“China can be centralized enough to compete with each individual US lab. They could have just as many flops in '25 and '26 if they decided they were scale built. Right? Just from foreign chips”
“Shanghai has anywhere from 45 to 50 high end immersion lithography tools is is what's like believed by intelligence as well as like many other folks.”
“funding the next node would not be economically viable anymore if it weren't for AI taking off. Right? And then generating all this humongous demand for the most leading edge chip.”
“There are 100 x gains from architecture. Even if we, like, literally stop shrinking, I think we could have 100 x gains from architectural advancements.”
“Microsoft slash OpenAI slash their partners for them. And then and then potentially even more, 500 k GB two hundreds, right, is a gigawatt. Right? And that's, like, online next year.”
“there's no fucking way you can pay for the scale of clusters that are being planned to be built next year for OpenAI until unless they raise, like, 50 to $100,000,000,000, which I think they will raise that, like, end of this year, early next year.”
“Like the R and D cost is much lower in terms of people, but the cost of goods sold in terms of actually operating the service, I think will be much higher.”
“Google's actually ramping up TPU production massively. And I think people in AI would be like, Well, duh. But like, okay, who has the capability of figuring out the number?”
“NVIDIA is gonna sell well over 3,000,000 total GPUs next year, over a million H100s this year alone. There's a lot of GPU capacity coming online. It's an incredible amount.”
“Hugging Face's libraries are actually very inefficient, like incredibly inefficient for inference. You get like 15% MBU on on on on some configurations like eight a one hundreds and Lama 70 b.”
“whole point was that Google you know, OpenAI, who everyone would be like, oh, yeah. They have more GPUs than anyone else. Right? But they have a lot less flops than Google.”
“there's like 10 companies that have enough compute in one single data center to be able to beat GPT-four. Right? Like straight up. Like, if not today, within the next six months.”
“NVIDIA is releasing a new chip, you know, in you know, they're gonna announce it in March and they're gonna release it, you know, and ship it, you know, q two, q three next year anyways. Right? And that chip will probably be three or four times as good.”
“But, like, the the level of the value that they deliver to the world, if you talk to anyone there, they truly believe it'll be tens of trillions if not hundreds of trillions of dollars.”
“And after GPT-five releases, if he goes to the market and says like, Hey, I want to raise $100,000,000,000 at $500,000,000,000 valuation, I'm sure the market would give it to him.”
“People don't understand how fragmented the semiconductor supply chain really is and how many monopolies there are. The US could absolutely shut shut down the Chinese semiconductor supply chain.”
“The export restrictions have clearly failed because China is still able to manufacture. And what's surprising about the chip that Huawei did is, you know, it's a phone chip.”
“SMIC just came out yesterday and said they're gonna ship 7,000,000 of these phones this year. Right? That is not low volume. That is very high volume, 7,000,000 phones.”
“TSMC and Intel were able to do it with what's called multi patterning DUV, deep ultraviolet. Right? So that just means they use the DUV lithography many times, right, for simplicity's sake.”
“So TSMC, to this day, even though they've been use they have over 50 EUV tools and they make hundreds of thousands of wafers a month with EUV, still uses DUV for their seven nanometer.”
“TSMC tried both. Right? And so at six nanometer and five nanometer, they use EUV, but they think cost efficiency wise, yield, right, which is how good the product's coming out, how many are good, how many need to be thrown away, the yield is better, the cost is better, the performance is better with DUV rather than EUV.”
“But remember the 1980i has continued to be upgraded and there's an upgraded version in 2023, right? And that tool, sort of the line is in between those tool, right?”
“If seven nanometer was completely economical, five nanometer, if the lithography element costs twice as much with the 1980i versus the next generation lithography tool, then that on the total chip basis doesn't cost anywhere close to that, right? Because again, the manufacturing cost of lithography is only one fourth to one fifth.”
“Because again, the manufacturing cost of lithography is only one fourth to one fifth. So if you double lithography costs, you're only increasing the total cost of a chip by 10%, right?”
“call it four or five five, six months between, you know, when an order is placed and you can actually have it installed in your data center if it got worked on immediately.”
“I like to joke, like, transformers are the most, like, capitalist thing that's ever been invented because you just throw more money at it, it gets better.”
“The bandwidth has not even gone up one order of magnitude. Right? Whereas flops have gone up two orders of magnitude. So so less than one order of magnitude versus two orders of magnitude increase.”
“So, you know, that that pricing has literally four x on the memory cost. And and now today, if you look at it, the cost of memory is 30 to 50% of the GPU cost.”
“And and now today, if you look at it, the cost of memory is 30 to 50% of the GPU cost. Great cost. And then obviously, NVIDIA does a four x markup on their”
“No one sells $4,000,000,000 a quarter of of of training GPU of of HPC GPUs. Right? They sell $4,000,000,000 a quarter of HPC AI GPUs. Right? So that's that's NVIDIA.”
“You know, you you have to you have to solidify on an open framework that supports multiple hardware vendors, but that open framework also needs to be the best way to use it on the existing incumbent NVIDIA hardware. That's the only way you could beat the monopoly.”
“@suchenzang There's a lot of evidence of organizations astro turfing false narratives about datacenter water use and noise, having funding from non domestic sources.
Not sure what he accusing me of tho?
I always tweet what I believe
Surely you're not saying I have some alterior motive”
“Incompetence in grid modeling is causing US ratepayers to pay $12B more than necessary.
The auction system of PJM is damaging to building more capacity.
The US needs market based reforms. https://t.co/Ghcuuz7ywJ”
“@damnang2 Our role is to track what's happening and provide updates as changes happen. Majority of my revenue is industry not hedge funds. Theyre the ones making actual useful decisions with the data.
All of these items we reported were true and we explained them. The one caveat is with”
“@insane_analyst @jwt0625 for the current things you've seen. There's a lot of cool research and companies beyond the 2 you have looked into. I think you should keep open mind to it, but agree theres not really any deployments of uled in short or medium term. The report says as much.”
“@insane_analyst @jwt0625 We literally did not treat it as equal. The note is comparing the two and says vcsel is better and more share. Bro what are these demons u fighting”
“@MichaelSte5069 @SemiAnalysis_ You’re gonna love our next newsletter :)
Unfortunately it’s hard to justify selling it to individuals, B2B is the only way we can justify the immense effort we have in it”
semianalysissemianalysis·posts only·
10 posts
·
0 of 0 appearances
new
post
“OpenAI Jalapeno is spicy
Usually first generation chips aren't competitive, but OpenAI is beating Nvidia Blackwell and even Rubin
This is huge news!
We got to go into OpenAI's lab to dissect their new chip and dive into the software, architecture, and performance
Incredible work! https://t.co/Q7Okgbxnp0”
“@damnang2 What's the solution?
Make every report a 2000 word salad that reexplains everything?
My firm puts out 15-20 reports a week about supply chain and technology for our clients.
We try to share clearly and consicely what's happening.
The people who boil it down to some silly trade, I”
“@damnang2 Our role is to track what's happening and provide updates as changes happen. Majority of my revenue is industry not hedge funds. Theyre the ones making actual useful decisions with the data.
All of these items we reported were true and we explained them. The one caveat is with”
“@insane_analyst @jwt0625 We literally did not treat it as equal. The note is comparing the two and says vcsel is better and more share. Bro what are these demons u fighting”
“@kospiev @phithetasigma False. It wasnt an expectation. It was me and Myron, the 3rd person at SemiAnalysis who wrote this. You obviously didnt read any the notes re timing either”
“@MichaelSte5069 @SemiAnalysis_ You’re gonna love our next newsletter :)
Unfortunately it’s hard to justify selling it to individuals, B2B is the only way we can justify the immense effort we have in it”
ai policyai policy·posts only·
11 posts
·
0 of 0 appearances
new
post
“@suchenzang There's a lot of evidence of organizations astro turfing false narratives about datacenter water use and noise, having funding from non domestic sources.
Not sure what he accusing me of tho?
I always tweet what I believe
Surely you're not saying I have some alterior motive”
“@joshrogin @pstAsiatech There's info everywhere about this. Talk to prior firms he's worked with.
I'll give you a tiny breadcrumb, a politico piece showing that his firm lobbies for China
https://t.co/SnM8y8w4Nz”
“The funny thing about Anthropic and OpenAI people saying they want to slow down AI progress is that this is what Anti Trust mechanisms were built for.
It's illegal to collude and slow down AI progress. https://t.co/YVfjrqG7Q3”
“Slowing down AI is a ultimately wishful thinking
The genie is out of the bottle
The most importan competition ever with potentially immeasurable benefits to the winner will not suddenly have people stop and sing kumbaya
While many lab employees have signed, many more havent. https://t.co/YVfjrqFA0v”
“@BronsonSchoen I read the letter and personally know dozens who signed this, and discussed with them IRL.
The letter brings no solution on how to do this or even attempts to”
“The funny thing about Anthropic and OpenAI people saying they want to slow down AI progress is that this is what Anti Trust mechanisms were built for.
It's illegal to collude and slow down AI progress. https://t.co/YVfjrqG7Q3”
“@BronsonSchoen I read the letter and personally know dozens who signed this, and discussed with them IRL.
The letter brings no solution on how to do this or even attempts to”
“Slowing down AI is a ultimately wishful thinking
The genie is out of the bottle
The most importan competition ever with potentially immeasurable benefits to the winner will not suddenly have people stop and sing kumbaya
While many lab employees have signed, many more havent. https://t.co/YVfjrqFA0v”
“OpenAI Jalapeno is spicy
Usually first generation chips aren't competitive, but OpenAI is beating Nvidia Blackwell and even Rubin
This is huge news!
We got to go into OpenAI's lab to dissect their new chip and dive into the software, architecture, and performance
Incredible work! https://t.co/Q7Okgbxnp0”
“The funny thing about Anthropic and OpenAI people saying they want to slow down AI progress is that this is what Anti Trust mechanisms were built for.
It's illegal to collude and slow down AI progress. https://t.co/YVfjrqG7Q3”
“@damnang2 Our role is to track what's happening and provide updates as changes happen. Majority of my revenue is industry not hedge funds. Theyre the ones making actual useful decisions with the data.
All of these items we reported were true and we explained them. The one caveat is with”
“Team just posted a note with the title "I can make your Bedrock" to discuss Amazon bedrock revenue mix to institional clients.
Incredible title https://t.co/CGBXaNoHhc”
“That $50,000,000,000 of COGS needs to burn on infra, which cost roughly with if a five if you're talking about five year depreciation, call it $250,000,000,000, right, of infra Yeah. For a $100,000,000,000 of revenue.”
“As we go forward into the future, the the numbers for computer ballooning, right, we're at, you know, you know, a little bit over a trillion dollars of CapEx this year.”
“That $50,000,000,000 of COGS needs to burn on infra, which cost roughly with if a five if you're talking about five year depreciation, call it $250,000,000,000,”
“As we go forward into the future, the the numbers for computer ballooning, right, we're at, you know, you know, a little bit over a trillion dollars of CapEx this year.”