Patel notes ARM took a decade from 2011 announcement to see large datacenter deployments in 2021.
“I think they announced in 2011 and we really only saw really big deployments from Amazon in 2021, right? Maybe 2020, right? Very big deployments.”
Patel says NVIDIA is shipping more GPU flops this year than in its entire data center history combined.
“There's more GPU flops shipping this year that NVIDIA shipped their entire history for the data center.”
Patel describes Microsoft's shift from 48-megawatt buildings to 300-megawatt AI-specific data centers for increased power density.
“Here you can see that same data center, that same type of data center vertically. And that's 48 megawatts, or 57, And then this data center center is like 300 megawatts.”
Patel reports Microsoft plans 1.5 gigawatts in one site within two and a half years across four 300 megawatt buildings.
“And so the scale that they wanna reach to, 1.5 gigawatts in two and a half years in one site.”
Patel reports OpenAI is going to Oracle for 200 megawatts because Microsoft cannot supply enough compute.
“OpenAI can't get enough compute from Microsoft, so then they're also going to Oracle. And Oracle's building another gigawatt scale data center eventually. Again, a gigawatt is like 500,000 plus GPUs.”
“And so the scale that they wanna reach to, 1.5 gigawatts in two and a half years in one site. Now the interesting thing is that each GPU takes roughly 1,000 watts to 1,400 watts.”
“Well, one, there's a gigawatt natural gas plant Yeah. That helps. Right next door. Right? Two, there's a natural gas line, a main that they tapped, and they set up their own generation capacity on-site.”
Patel forecasts data center power will grow from 150 megawatts today to gigawatt-scale clusters by 2026.
“We're seeing, like, you know, 400, 500 megawatts cluster sort of being built out now. Right? Like you know? And and then, you know, in 2026, I'd imagine it'll be, like, gigawatt scale clusters.”
Patel describes CoreWeave converting a crypto facility with 200 megawatt gas plant into seven data center buildings.
“And one of these crypto data centers has a, like, a 200 megawatt natural gas plant in the middle of, like, seven buildings. Right? And each of these buildings is a data center.”
Patel says Oracle is spending over $10 billion this year building infrastructure for OpenAI, not Microsoft.
“Oracle's spending, like, $10,000,000,000 plus for them this year to build out data centers and GPUs, right, for OpenAI, not Microsoft, interestingly enough.”
“these companies are building three gigawatts of data center capacity. Put that in context, at the beginning of twenty twenty four, that was roughly Meta's global footprint.”
Patel says Malaysia will add three gigawatts of data center capacity by 2027, mostly Chinese companies claiming to be Singaporean.
“Malaysia from 2024 to 2027, not the country itself, but companies operating there, mostly Chinese companies, many of them claiming they're now Singaporean companies. Right?”
Patel says Chinese data center companies are building three gigawatts in Malaysia by 2027, claiming Singaporean status.
“Like, the largest operator in China of data centers in China, GDS, moved to Singapore and says they're Singaporean now. It's like but now they're these companies are building three gigawatts of data center capacity.”
Patel says Meta is building two gigawatts of capacity in Louisiana powered by natural gas.
“They they uncensored politics. There's a bit of massive vibe shift. One of the things they're doing is, like, they're setting up two gigawatts.”
Patel identifies a 40 gigawatt power shortfall for US data centers even with all planned capacity additions.
“There is an over 40 gigawatt hole in power for The US. Even if we don't turn off any more coal, in fact we restart some, we take all the natural gas dual combined cycle reactor deliveries that we're going to have, and you add in renewables, you have a 40 gigawatt hole of power production in The US.”
Patel identifies a 40 gigawatt power deficit in US even with all planned gas and restarted coal.
“Even if we don't turn off any more coal, in fact we restart some, we take all the natural gas dual combined cycle reactor deliveries that we're going to have,”
Patel says certain states will see over half their power consumed by AI within a few years.
“When we look at power consumption by state, something pretty simple, there are certain states where more than half their power is going be consumed by AI just in the next few years.”
Patel says CoreWeave can build $10 billion clusters from start to completion much faster than competitors.
“it lets you get to these, you know, call it $10,000,000,000 clusters, right, in in time scales, from project start to completion that are much much shorter than anyone else.”
Patel observes hyperscaler data centers started in 2023 remain incomplete even with expedited equipment delivery.
“Even if they even if they had the electrical equipment in in you know, not on lead time, they paid a bunch to skip the queue.”
Patel describes former Yandex workers paid bonuses for speed who use drugs to complete data center buildouts faster.
“Like and they get paid bonuses for being faster, and therefore, do, like, certain drugs to be able to finish the build outs faster.”
Patel reveals Meta is building temporary tent structures to house GPUs because permanent buildings take too long to construct.
“Meta is literally building these, like, temporary, like, tent structures to put GPUs in because building the building takes too long, and it takes too much labor, as you mentioned labor.”
Patel says GPU hardware and networking represent 80% of data center costs, with power and cooling only 20%.
“It's the GPU purchases. It's the networking. It's the it's the physical data center conversion power conversion equipment. All of this stuff is, like, 80% of the cost.”
Patel suggests NVIDIA should use its massive tax bill to invest directly in data center infrastructure despite customer conflicts.
“Now this is obviously gonna be, like, crazy because, like, now they're buying g p their own GPUs and putting them in data centers and doing stuff, and they're competing with their own customers,”
Patel states 80% of GPU data center cost is capital equipment, only 20% is land, power, and cooling.
“It's the it's the physical data center conversion power conversion equipment. All of this stuff is, like, 80% of the cost. And then 20% is gonna be your land and your power and your cooling”
Patel notes new NVIDIA servers require 140 kilowatts per rack versus prior CPU data centers at kilowatts.
“Data centers for CPUs, 10 megawatts of rack, 12 megawatts of rack. Kilowatts. The new ones, kilowatts, right? Kilowatts of rack, sorry. The new NVIDIA servers, 140 kilowatts of rack.”
Patel explains Oracle's $300 billion OpenAI bet is lower risk because Oracle only secures data centers, not expensive GPUs upfront.
“Now, of course, the bet is a bit like there's a bit more security in the bet in that Oracle really only needs to secure the data center capacity.”
Patel notes GB200 expanded NVLink from 8 to 72 GPUs and rack power jumped from 10 to 140 kilowatts.
“Now you have 72 GPUs. And if you go look at the rack, right? It's completely different, right? It's liquid cooled. It's an entire rack that consumes 140 kilowatts, whereas H100 servers consumed 10 kilowatts.”
Patel describes Meta's temporary tent-like data centers designed for single GPU generation lifecycles instead of 30-year buildings.
“Meta has got a data center design that's only built to last for a single generation. They put them in these tent buildings.”
Patel contrasts traditional data centers with minimal retrofit costs against modern GPU deployments requiring major upgrades each generation.
“it used to be you'd build a data center and there wouldn't really be much retrofits cost, right? You would slide servers in and out, you would rent racks to people.”
Patel contrasts GB200 rack consuming 140 kilowatts with H100 servers at 10 kilowatts each.
“It's liquid cooled. It's an entire rack that consumes 140 kilowatts, whereas H100 servers consumed 10 kilowatts. And you might put two, you might put four in a rack, right? And it was air cooled.”
Nadella describes connecting multiple Fairwater facilities on a petabit network spanning to Milwaukee for distributed AI training.
“Fairwater 4, which you're going to see under construction nearby, will also be on that one petabits network so that we can actually link the two at a very high rate.”
Patel says data center power capacity will nearly double from 15-18 gigawatts this year to 30 gigawatts next year.
“next year, 30 gigawatts are being added and we think the power is there for it. Wow.”
Patel says data center power capacity is growing from 15-18 gigawatts this year to 30 gigawatts next year.
“What was it It's this it's or this year is like I think it's like 18 ish, 10 ish. Okay. Fifteen fifteen to 18”
Patel says data centers will reach 10% of US power by 2027-28 but under 1% of water consumption.
“US Grid will get to like 10% of power by like '28, 27 is data centers. For water consumption, it's not even gonna crack 1%. Yeah. By the end of the decade.”
Patel forecasts 20 gigawatts of data center capacity in 2024 and over 30 gigawatts in 2025 despite delays.
“This year there's going to be 20 gigawatts, even accounting for the delays, and next year there's going be more than 30 gigawatts accounting for the delays.”
Patel forecasts data center capacity will reach 30 gigawatts in the next year.
“This year, we're deploying 20 gigawatts of data centers. Next year, that number goes up 50 gigawatts or 50%, sorry. 50%, sorry.”
Patel says a data center is physically redoing doors to accommodate the double-width Helios racks.
“There's a there's a data center that I know of specifically that is going to be deploying Helios, and they are redoing the doors so that they can reel the Helios in.”
Patel says SemiAnalysis has more China compute mapped than cited estimate
“@RothRottweiler @aqib_zakaria @hamandcheese @SemiAnalysis_ Idk about all that chief. We have more compute for China mapped than this”