The Minutes of Dylan Patel on inference economics https://minutesof.com/dylan-patel/on/inference-economics/ Everything Dylan Patel has said on inference economics: 30 verbatim quotes between December 2023 and August 2026, each with a timestamp and a link to the… en Sun, 30 Aug 2026 16:22:16 +0000 Patel says Anthropic generates $50 million per megawatt revenue, enabling 5x return on inference spending. https://minutesof.com/q/6e3f2d04-6524-4dca-bc2d-340a73646ca2/ https://minutesof.com/q/6e3f2d04-6524-4dca-bc2d-340a73646ca2/ “In the case of Anthropic, the the revenue has gone as high as $50,000,000 per megawatt.” — Dwarkesh Patel Tue, 25 Aug 2026 15:57:53 +0000 inference economics Patel reveals Anthropic turned first profit in Q2 and will hit billion-dollar operating profit in Q3 before IP https://minutesof.com/q/ab25a556-53cd-4e2e-ba7e-2336d5d78b64/ https://minutesof.com/q/ab25a556-53cd-4e2e-ba7e-2336d5d78b64/ “So Anthropic turned their first gross profit in Q2 in June. And then in Q3, they will be turning a billion dollars of operating profit, slightly over.” — RAISE Summit Thu, 16 Jul 2026 16:44:57 +0000 inference economics Patel reports OpenAI gross margins rose from 30% to 55% overall, 50% to 65% excluding free users. https://minutesof.com/q/552929f1-c8ae-43c3-8501-0ab7970989ce/ https://minutesof.com/q/552929f1-c8ae-43c3-8501-0ab7970989ce/ “You look at OpenAI late last year, their margins had were roughly 30% gross margin, but if you stripped away the free users, they were at 50%.” — RAISE Summit Thu, 16 Jul 2026 16:44:57 +0000 inference economics Patel says their AI spending equals over a third of employee compensation for their 90-person firm. https://minutesof.com/q/6f7b1cde-3481-4cd5-89a6-c38b8939b845/ https://minutesof.com/q/6f7b1cde-3481-4cd5-89a6-c38b8939b845/ “We're spending, you know, more like, you know, more than a third of the spend, you know, employee spend, a third of it is also on top of that is AI.” — WisdomTree in Europe Thu, 09 Jul 2026 06:16:39 +0000 inference economics Patel argues Mythos fast mode is probably cheaper than Claude 4.6 fast mode for most tasks due to token effici https://minutesof.com/q/9d03e1ee-9f3e-449f-bbd7-ef96c09864e7/ https://minutesof.com/q/9d03e1ee-9f3e-449f-bbd7-ef96c09864e7/ “the flip side is, is, is mythos is more token efficient. Methos fast mode is probably cheaper than like, four, six fast mode for most tasks.” — SemiAnalysis Weekly Wed, 06 May 2026 00:55:06 +0000 inference economics Patel cites Anthropic adding $67 billion ARR per month as evidence of expanding AI revenue beyond hyperscalers https://minutesof.com/q/8d242968-225b-41c7-86f6-4f27678a4907/ https://minutesof.com/q/8d242968-225b-41c7-86f6-4f27678a4907/ “You're starting to see it with Anthropix revenue adding $67,000,000,000 of ARR a month, but there's so many more firms coming online with revenue streaming in,” — CNBC Television Mon, 16 Mar 2026 21:58:55 +0000 inference economics Patel states Anthropic now adds $23 billion revenue monthly versus few hundred million earlier. https://minutesof.com/q/a9df3580-18d1-4ac3-9f9d-afcb758ea6d7/ https://minutesof.com/q/a9df3580-18d1-4ac3-9f9d-afcb758ea6d7/ “Anthropics, you know, adding $23,000,000,000 of revenue a month now Mhmm. Versus they were just adding a few 100,000,000 of revenue a month earlier. So clearly, we're in the take off period.” — Latent Space Thu, 26 Feb 2026 21:15:18 +0000 inference economics Patel reports Claude Code doubled to 4% of GitHub commits in January alone, with overall AI coding likely at 1 https://minutesof.com/q/d7ab6afb-29cc-4e7c-9868-7f4eb011caa2/ https://minutesof.com/q/d7ab6afb-29cc-4e7c-9868-7f4eb011caa2/ “Just in this month just in January, it went from 4% of or 2% of commits on GitHub to 4% of GitHub commits were done by Cloud Code.” — Latent Space Thu, 26 Feb 2026 21:15:18 +0000 inference economics Patel explains prefill costs one-fourth of decode per token, making 30,000 token inputs costlier than 2-4,000 https://minutesof.com/q/187d2021-25f1-4e4f-8559-763189933ea6/ https://minutesof.com/q/187d2021-25f1-4e4f-8559-763189933ea6/ “And that's a very common ratio. Right? And then when you think about, okay, the cost of running pre fill is roughly one fourth of running decode.” — Clockwork Fri, 21 Nov 2025 17:18:51 +0000 inference economics Patel states avoiding redundant prefill through KV cache can cut inference costs to one-fourth of previous lev https://minutesof.com/q/c93c6e47-fe3d-405d-b28b-2d99a4919dee/ https://minutesof.com/q/c93c6e47-fe3d-405d-b28b-2d99a4919dee/ “So you can cut your cost to one fourth of what it was previously if you just don't do the pre fill. Right?” — Clockwork Fri, 21 Nov 2025 17:18:51 +0000 inference economics Patel says inference providers sell public endpoints at flat or negative margins, compensating through private https://minutesof.com/q/a5a36fd6-ef64-421d-871a-a04382e1453c/ https://minutesof.com/q/a5a36fd6-ef64-421d-871a-a04382e1453c/ “Most of the inference providers are selling at flat margins or even negative for their public endpoints. And they then make it up when people do private deployments.” — Open Compute Project Thu, 23 Oct 2025 06:01:45 +0000 inference economics Patel calculates 20% power efficiency difference translates to only 4% TCO difference on NVIDIA deployments. https://minutesof.com/q/e54db0c9-c69e-4b1e-a270-5018b18ad51e/ https://minutesof.com/q/e54db0c9-c69e-4b1e-a270-5018b18ad51e/ “Because if you have enough power, a 20% difference in performance per watt only ends up being a 4% difference in TCO.” — Open Compute Project Thu, 23 Oct 2025 06:01:45 +0000 inference economics Patel says the standard inference deployment unit has shifted from single nodes to hundreds of GPUs. https://minutesof.com/q/ee180f12-a142-4bbb-8e53-e032a9458215/ https://minutesof.com/q/ee180f12-a142-4bbb-8e53-e032a9458215/ “the standard unit for an inference deployment being hundreds of GPUs instead of a single node. And then there's all these different things about traffic.” — Open Compute Project Thu, 23 Oct 2025 06:01:19 +0000 inference economics Patel notes NVIDIA Blackwell performance improved dramatically from launch to present, as did AMD hardware. https://minutesof.com/q/8a048f38-2c8d-4690-a1d2-66e094aaa6fb/ https://minutesof.com/q/8a048f38-2c8d-4690-a1d2-66e094aaa6fb/ “If you tried to use NVIDIA's Blackwell six months ago, the numbers were not amazing, right? But now they're actually amazing. So how did that progress over time? Same with AMD, right?” — Open Compute Project Thu, 23 Oct 2025 06:01:19 +0000 inference economics Patel breaks down inference TCO as 20% electricity and data center, 80% hardware on NVIDIA deployments. https://minutesof.com/q/3358ce5d-fe0f-496b-a2f4-13edb43515e9/ https://minutesof.com/q/3358ce5d-fe0f-496b-a2f4-13edb43515e9/ “20% of your cost is your electricity, your data center real estate, roughly. And then the rest of the cost is that hardware, at least on a standard NVIDIA deployment your GPUs, your networking, etcetera.” — Open Compute Project Thu, 23 Oct 2025 06:01:19 +0000 inference economics Patel identifies GB200's power efficiency advantage but notes deployment challenges with backplane and liquid https://minutesof.com/q/d7dccc2a-7013-4972-b5c9-372d97cae003/ https://minutesof.com/q/d7dccc2a-7013-4972-b5c9-372d97cae003/ “GB200 has a huge power efficiency advantage, right? Everyone here understands the challenges of running and deploying GB200. There's a lot of challenges with the backplane.” — Open Compute Project Thu, 23 Oct 2025 06:01:19 +0000 inference economics Patel reports GB200 is 10x more power efficient than H200 at certain interactivity rates. https://minutesof.com/q/70c03321-ad13-4824-91b8-9ec2ebd3b7be/ https://minutesof.com/q/70c03321-ad13-4824-91b8-9ec2ebd3b7be/ “But it turns out at certain interactivity rates, I. E. Tokens per second per user, it's 10x more efficient per watt, right, compared to H200.” — Open Compute Project Thu, 23 Oct 2025 06:01:19 +0000 inference economics Patel finds AMD MI355 beats NVIDIA B200 on performance TCO in certain publicly usable configurations. https://minutesof.com/q/b9346a92-e689-4ff1-a8aa-a519c318c01e/ https://minutesof.com/q/b9346a92-e689-4ff1-a8aa-a519c318c01e/ “So we do different scenarios. We do document processing, which is 8,000 context in, 1,000 out. We do chat, which is 1,000 in, 1,000 out.” — Open Compute Project Thu, 23 Oct 2025 06:01:19 +0000 inference economics Patel shows B200 has 15x raw performance advantage over H100 but only 10x performance per TCO. https://minutesof.com/q/101225bc-685d-4ce2-a64b-7e2df33b1f98/ https://minutesof.com/q/101225bc-685d-4ce2-a64b-7e2df33b1f98/ “If we don't divide by TCO, then it looks like the performance of B200 is actually 15x that of H100, versus the performance TCO is only 10x,” — Open Compute Project Thu, 23 Oct 2025 06:01:19 +0000 inference economics Patel states Supermicro's liquid cooling reduces Hopper server power from 10 kilowatts to 7-8 kilowatts versus https://minutesof.com/q/98989f27-8eb3-4a61-b86b-0dd09799deee/ https://minutesof.com/q/98989f27-8eb3-4a61-b86b-0dd09799deee/ “Everyone else's hopper servers h 100 air cooled. And so each server consumes, you know, 10 kilowatts almost. Right?” — Supermicro Thu, 16 Oct 2025 17:00:47 +0000 inference economics Patel calculates that without Supermicro's liquid cooling solution, servers would consume 30% more power. https://minutesof.com/q/10a69170-989c-42e0-9bd5-da11329eee97/ https://minutesof.com/q/10a69170-989c-42e0-9bd5-da11329eee97/ “So that's why Shibbol Micro dedicated deep cooling so aggressively. And if the liquid cooling solution from Super Micro didn't exist, then those servers would be consuming 30% more power.” — Supermicro Thu, 16 Oct 2025 17:00:47 +0000 inference economics Patel describes diagnostic challenges in AI data centers where failures occur across multiple infrastructure l https://minutesof.com/q/b7a7f145-e89d-4c61-b6f7-3bea9c8248c6/ https://minutesof.com/q/b7a7f145-e89d-4c61-b6f7-3bea9c8248c6/ “And people have had problems where their data center wasn't ready and something stopped working. They don't know, is it the chip? No.” — Supermicro Thu, 16 Oct 2025 17:00:47 +0000 inference economics Patel explains OpenAI's router will send low-value queries to cheaper models but use expensive compute for hig https://minutesof.com/q/0502aa27-d821-4087-8f4b-18a954c32235/ https://minutesof.com/q/0502aa27-d821-4087-8f4b-18a954c32235/ “if the user asks a low value query like, hey, why is the sky blue? Just route them to mini. The model can answer perfectly fine, and that is a chunk of queries.” — a16z Mon, 18 Aug 2025 18:26:43 +0000 inference economics Patel says AI model costs dropped 1200x from GPT-3's $60 per million outputs to current pricing. https://minutesof.com/q/41d2ba76-f8de-4943-bca0-4e696568c0ea/ https://minutesof.com/q/41d2ba76-f8de-4943-bca0-4e696568c0ea/ “GPT three was it cost, you know, it cost $60 for the million output. And over time, that kept reducing. Right? OpenAI released new models.” — MedBricks Webcast Thu, 27 Mar 2025 13:55:08 +0000 inference economics Patel explains reasoning models dramatically increase memory usage and reduce batch size, multiplying serving https://minutesof.com/q/fb69a564-812e-47e6-9f3d-e6f80b0878ca/ https://minutesof.com/q/fb69a564-812e-47e6-9f3d-e6f80b0878ca/ “So your your memory usage is going way up with these reasoning models, and you still have a lot of users.” — Lex Fridman Mon, 03 Feb 2025 00:12:13 +0000 inference economics Patel explains reasoning models dramatically increase serving costs due to long output context and memory cons https://minutesof.com/q/8ea24247-feff-4e7f-beec-571f6109d01d/ https://minutesof.com/q/8ea24247-feff-4e7f-beec-571f6109d01d/ “your memory usage is going way up with these reasoning models, and you still have a lot of users. So effectively, the cost to serve multiplies by a ton.” — Lex Fridman Mon, 03 Feb 2025 00:12:13 +0000 inference economics Patel estimates Blackwell delivers 10-15x cost improvement for inference despite NVIDIA claiming 30x at GTC. https://minutesof.com/q/879abc84-3fdc-4dd7-9b63-baf6f6541b7c/ https://minutesof.com/q/879abc84-3fdc-4dd7-9b63-baf6f6541b7c/ “But now, like, Blackwell, NVIDIA's pitching 10 to 15 x improvement in cost. It's like, well, you know, they're massaging the numbers marketing.” — Unsupervised Learning: With Jacob Effron Tue, 21 Jan 2025 14:00:12 +0000 inference economics Patel explains OpenAI's o1 model thinks for 5-20 seconds before outputting, creating inference throughput chal https://minutesof.com/q/179104b9-35d2-44c2-9d8a-85d97e3bd8f8/ https://minutesof.com/q/179104b9-35d2-44c2-9d8a-85d97e3bd8f8/ “When you use OpenAI's o one, it thinks for ten seconds, twenty seconds, five seconds. It varies a lot, but thinks for a while and then it sends you tokens.” — Scaling Intelligence Tue, 12 Nov 2024 04:17:00 +0000 inference economics Patel reports H100 rental prices have dropped from $3-4 per hour to $2.15 or less, approaching natural cost of https://minutesof.com/q/e377715e-4aba-45ba-9b19-f37de850e568/ https://minutesof.com/q/e377715e-4aba-45ba-9b19-f37de850e568/ “An hour. Right? For shorter term or midterm deals. Right now, it's like, if you want a six month deal, you could get, like, $2.15 or less.” — Dwarkesh Patel Wed, 02 Oct 2024 14:59:36 +0000 inference economics Patel argues AI software will have lower R&D costs but much higher cost of goods sold from operating services. https://minutesof.com/q/cdca4ea2-53cc-42c9-a237-c2927fb44bb3/ https://minutesof.com/q/cdca4ea2-53cc-42c9-a237-c2927fb44bb3/ “Like the R and D cost is much lower in terms of people, but the cost of goods sold in terms of actually operating the service, I think will be much higher.” — Latent Space Tue, 05 Dec 2023 18:16:28 +0000 inference economics