https://minutesof.com/on/inference/ Everyone on the record about inference: 75 verbatim quotes from 5 people, November 2019 to August 2026, in one chronology, each with a timestamp and a… en Sun, 30 Aug 2026 22:33:01 +0000 https://minutesof.com/q/469a18e6-8605-46bd-bea0-fa21bd5502b3/ https://minutesof.com/q/469a18e6-8605-46bd-bea0-fa21bd5502b3/ “So, this is sort of what I personally believe that the labs are going to allocate less and less compute to inference over time, I think is very non consensus, right?” — Dylan Patel, Dwarkesh Patel Tue, 25 Aug 2026 15:57:53 +0000 inference https://minutesof.com/q/fefa3320-9068-4e30-84da-c70370affef0/ https://minutesof.com/q/fefa3320-9068-4e30-84da-c70370affef0/ “Go download the Kimi weights. Go download VLM or SGLANG. Set it up. You know, Codecs and Fable can actually help you do this.” — Dylan Patel, Dwarkesh Patel Tue, 25 Aug 2026 15:57:53 +0000 inference https://minutesof.com/q/ccbeb5bf-7c6b-4af0-ae7d-f4c729f341ed/ https://minutesof.com/q/ccbeb5bf-7c6b-4af0-ae7d-f4c729f341ed/ “If model progress at the labs pause, then more compute comes online. It has to slow down. Right? Sort of right now we have supply demand, right?” — Dylan Patel, SemiAnalysis Mon, 17 Aug 2026 15:00:06 +0000 inference https://minutesof.com/q/898427d9-5d8c-40af-9feb-9256d6c3c56e/ https://minutesof.com/q/898427d9-5d8c-40af-9feb-9256d6c3c56e/ “And they're monetizing at something like 50,000,000,000 a gig and consensus estimates for next year are 73,000,000,000. So forget Starlink v three, forget Starlink direct to sell, Grok 4.5 and Cursor.” — Gavin Baker, Invest Like the Best Tue, 04 Aug 2026 08:00:00 +0000 inference https://minutesof.com/q/273fa3f1-13d3-4f0d-9774-ac10ff1c4250/ https://minutesof.com/q/273fa3f1-13d3-4f0d-9774-ac10ff1c4250/ “The amount of compute is going up and inference margins going up. And if you multiply those three, that's how you're getting this crazy acceleration into some of the labs plus open source,” — Gavin Baker, Invest Like the Best Tue, 04 Aug 2026 08:00:00 +0000 inference https://minutesof.com/q/273fa3f1-13d3-4f0d-9774-ac10ff1c4250/ https://minutesof.com/q/273fa3f1-13d3-4f0d-9774-ac10ff1c4250/ “The amount of compute is going up and inference margins going up. And if you multiply those three, that's how you're getting this crazy acceleration into some of the labs plus open source,” — Gavin Baker, Invest Like the Best Tue, 04 Aug 2026 08:00:00 +0000 inference https://minutesof.com/q/443a6457-4940-4dff-9022-a9659a14fe37/ https://minutesof.com/q/443a6457-4940-4dff-9022-a9659a14fe37/ “They went on a podcast, and they essentially said, we are planning to pay 100% more for Blackwell's when our contract expires. And that just means that essentially all the hyperscalers are under earning.” — Gavin Baker, Invest Like the Best Tue, 04 Aug 2026 08:00:00 +0000 inference https://minutesof.com/q/2f00fe00-eb15-4fb6-83b8-ad9eee2d466d/ https://minutesof.com/q/2f00fe00-eb15-4fb6-83b8-ad9eee2d466d/ “And they had rented a cluster of several thousand black wells, and we'll just call it somewhere in the mid $2 per GPU hour.” — Gavin Baker, Invest Like the Best Tue, 04 Aug 2026 08:00:00 +0000 inference https://minutesof.com/q/c85f4400-5d03-4327-a817-cfa56c88e0a0/ https://minutesof.com/q/c85f4400-5d03-4327-a817-cfa56c88e0a0/ “And so, essentially, you have the contracted base of installed compute trading at a massive discount to the current spot market.” — Gavin Baker, Invest Like the Best Tue, 04 Aug 2026 08:00:00 +0000 inference https://minutesof.com/q/12c60899-9cef-4341-915a-6afdeefc7d8d/ https://minutesof.com/q/12c60899-9cef-4341-915a-6afdeefc7d8d/ “we're seeing cache hit rates above 95% for many AgenTeq workflows, which which means the cost for a cache hit is it's not free,” — Dylan Patel, RAISE Summit Thu, 16 Jul 2026 16:44:57 +0000 inference https://minutesof.com/q/3bea5703-d869-4e93-bec6-70e3ecd20e14/ https://minutesof.com/q/3bea5703-d869-4e93-bec6-70e3ecd20e14/ “you calculate that once, you store it off in memory, whether it be system memory or storage, and then you pull it back in when you run the turn.” — Dylan Patel, RAISE Summit Thu, 16 Jul 2026 16:44:57 +0000 inference https://minutesof.com/q/a3014961-4f96-48f2-8af1-0b0ad506d194/ https://minutesof.com/q/a3014961-4f96-48f2-8af1-0b0ad506d194/ “And and that makes, you know, initially, that'd be like, okay, well now I need a ton ton of compute to calculate all the prefilled tokens.” — Dylan Patel, RAISE Summit Thu, 16 Jul 2026 16:44:57 +0000 inference https://minutesof.com/q/92c8f11c-3e80-4ba8-9041-57c9ce86cb86/ https://minutesof.com/q/92c8f11c-3e80-4ba8-9041-57c9ce86cb86/ “Monopoly Go is a game where we have the revenue and the hours played and it is to render it using list prices for something like VEO3 is more than two orders of magnitude greater than its revenue. So what is the role for human creativity, man? I don't know.” — Gavin Baker, Generating Alpha Podcast Wed, 08 Jul 2026 15:54:53 +0000 inference https://minutesof.com/q/f9d8be14-a3e0-4d46-93f0-a74b86319592/ https://minutesof.com/q/f9d8be14-a3e0-4d46-93f0-a74b86319592/ “AI, even if you put a harness on it, even if you do the chain of thought, even if you have multiple agents, it's probabilistic that it is recomputing the answer each time.” — Gavin Baker, Generating Alpha Podcast Wed, 08 Jul 2026 15:54:53 +0000 inference https://minutesof.com/q/e9a8fba3-1f79-4310-9c95-0d513f20374b/ https://minutesof.com/q/e9a8fba3-1f79-4310-9c95-0d513f20374b/ “by 2030, just OpenAI and Anthropic will have over 100 gigawatts combined, and then you'll add Meta and Google and so on and so forth.” — Dylan Patel, Sequoia Capital Tue, 30 Jun 2026 12:00:25 +0000 inference https://minutesof.com/q/dd349bfc-a119-4399-bf3c-9a26efbef20a/ https://minutesof.com/q/dd349bfc-a119-4399-bf3c-9a26efbef20a/ “I think by 2030, just OpenAI and Anthropic will have over 100 gigawatts combined, and then you'll add Meta and Google and so on and so forth.” — Dylan Patel, Sequoia Capital Tue, 30 Jun 2026 12:00:25 +0000 inference https://minutesof.com/q/d97e5674-e5aa-447d-9755-f683cbf66ab1/ https://minutesof.com/q/d97e5674-e5aa-447d-9755-f683cbf66ab1/ “The useful life of GPU is only a year or two. The useful life of CPU is only four years because the rapid technological change.” — Gavin Baker, Invest Like the Best Wed, 20 May 2026 08:00:00 +0000 inference https://minutesof.com/q/d42a1b86-60fc-431a-8a6b-892fc8405b51/ https://minutesof.com/q/d42a1b86-60fc-431a-8a6b-892fc8405b51/ “The disaggregation of inference means that I think these GPUs are going to have ten or fifteen year lives. The AI skeptics are like, oh, these companies are all cooking their books.” — Gavin Baker, Invest Like the Best Wed, 20 May 2026 08:00:00 +0000 inference https://minutesof.com/q/d42a1b86-60fc-431a-8a6b-892fc8405b51/ https://minutesof.com/q/d42a1b86-60fc-431a-8a6b-892fc8405b51/ “The disaggregation of inference means that I think these GPUs are going to have ten or fifteen year lives. The AI skeptics are like, oh, these companies are all cooking their books.” — Gavin Baker, Invest Like the Best Wed, 20 May 2026 08:00:00 +0000 inference https://minutesof.com/q/ca1f41f5-e05d-4029-a1db-504f8b4e433d/ https://minutesof.com/q/ca1f41f5-e05d-4029-a1db-504f8b4e433d/ “The disaggregation of inference means that I think these GPUs are going to have ten or fifteen year lives.” — Gavin Baker, Invest Like the Best Wed, 20 May 2026 08:00:00 +0000 inference https://minutesof.com/q/ca1f41f5-e05d-4029-a1db-504f8b4e433d/ https://minutesof.com/q/ca1f41f5-e05d-4029-a1db-504f8b4e433d/ “The disaggregation of inference means that I think these GPUs are going to have ten or fifteen year lives.” — Gavin Baker, Invest Like the Best Wed, 20 May 2026 08:00:00 +0000 inference https://minutesof.com/q/b32b5e45-ab6f-4748-bcad-62727d94ff2f/ https://minutesof.com/q/b32b5e45-ab6f-4748-bcad-62727d94ff2f/ “Continual learning is a model that dynamically adjusts its weights or adjusts in some way in real time.” — Gavin Baker, Invest Like the Best Wed, 20 May 2026 08:00:00 +0000 inference https://minutesof.com/q/e54157aa-cef1-4bd4-9d22-a8b4df8c0d13/ https://minutesof.com/q/e54157aa-cef1-4bd4-9d22-a8b4df8c0d13/ “I think the shift to usage based pricing is probably why you will see OpenAI and Anthropic exceed well over $200,000,000,000 in ARR this year.” — Gavin Baker, Invest Like the Best Wed, 20 May 2026 08:00:00 +0000 inference https://minutesof.com/q/60e611b0-5da1-46dd-88cc-3277d267722f/ https://minutesof.com/q/60e611b0-5da1-46dd-88cc-3277d267722f/ “So I think the shift to usage based pricing is probably why you will see OpenAI and Anthropic exceed well over $200,000,000,000 in ARR this year.” — Gavin Baker, Invest Like the Best Wed, 20 May 2026 08:00:00 +0000 inference https://minutesof.com/q/4b52026f-d927-4a34-90b9-14245a9cc937/ https://minutesof.com/q/4b52026f-d927-4a34-90b9-14245a9cc937/ “AI is just shifting from all you can eat to pay by the drink. Then it turns out people really like to talk to their friends long distance.” — Gavin Baker, Invest Like the Best Wed, 20 May 2026 08:00:00 +0000 inference https://minutesof.com/q/7b49081d-b3f9-445d-aba4-d290d5374ebd/ https://minutesof.com/q/7b49081d-b3f9-445d-aba4-d290d5374ebd/ “To understand what Frontier AI is capable of today, even for a non coding use case, need to have Cloud Code or Codex five point Codex. And you need to be on an enterprise plan.” — Gavin Baker, Invest Like the Best Wed, 20 May 2026 08:00:00 +0000 inference https://minutesof.com/q/42fa040e-9f02-4acf-911f-dfbc9a85a96b/ https://minutesof.com/q/42fa040e-9f02-4acf-911f-dfbc9a85a96b/ “To understand what Frontier AI is capable of today, even for a non coding use case, need to have Cloud Code or Codex five point Codex.” — Gavin Baker, Invest Like the Best Wed, 20 May 2026 08:00:00 +0000 inference https://minutesof.com/q/7991d993-a149-4684-8692-12d7dc932f1e/ https://minutesof.com/q/7991d993-a149-4684-8692-12d7dc932f1e/ “And it turns out that harness engineering is not as important as the model, but it really matters. And these harnesses in these models are increasingly being co developed.” — Gavin Baker, Invest Like the Best Wed, 20 May 2026 08:00:00 +0000 inference https://minutesof.com/q/c36e87b2-cd1e-4c19-b84d-b929d673346d/ https://minutesof.com/q/c36e87b2-cd1e-4c19-b84d-b929d673346d/ “It's why I'm probably less worried about like an edge AI bear case than I was. We're going to consume as much compute as we can.” — Gavin Baker, Invest Like the Best Wed, 20 May 2026 08:00:00 +0000 inference https://minutesof.com/q/c36d1b7b-a557-4ecb-b855-6610596f9fe3/ https://minutesof.com/q/c36d1b7b-a557-4ecb-b855-6610596f9fe3/ “And I just think the same is true of compute. It's why I'm probably less worried about like an edge AI bear case than I was.” — Gavin Baker, Invest Like the Best Wed, 20 May 2026 08:00:00 +0000 inference https://minutesof.com/q/ec5a9c7e-99d9-4536-8535-fbba88740844/ https://minutesof.com/q/ec5a9c7e-99d9-4536-8535-fbba88740844/ “I think Anthropic probably starts generating cash this year if they are not already generating cash, which I think is probably the case.” — Gavin Baker, Invest Like the Best Wed, 20 May 2026 08:00:00 +0000 inference https://minutesof.com/q/52541184-dd18-482e-b266-0236fe4216b1/ https://minutesof.com/q/52541184-dd18-482e-b266-0236fe4216b1/ “And I think maybe a true statement is that Infantropic could just wave a magic wand and get all the compute they wanted. They'd probably be doing well north of $100,000,000,000 today, maybe 150.” — Gavin Baker, Invest Like the Best Wed, 20 May 2026 08:00:00 +0000 inference https://minutesof.com/q/108fa4fc-95f6-46b6-b83f-019d063ae52f/ https://minutesof.com/q/108fa4fc-95f6-46b6-b83f-019d063ae52f/ “There is no intelligence. There's no consumer chat GPT. There's no enterprise intelligence. There's no clawed code without the production of tokens.” — Brad Gerstner, Altimeter Capital Fri, 15 May 2026 20:31:59 +0000 inference https://minutesof.com/q/4bbf9d84-bae2-460e-a752-a60a13663534/ https://minutesof.com/q/4bbf9d84-bae2-460e-a752-a60a13663534/ “I could've Atreides could've invested over $50,000,000 in the round at 1,100,000,000, And I was conflicted out by Crusoe,” — Gavin Baker, Sohn Conference Foundation Fri, 15 May 2026 18:17:41 +0000 inference https://minutesof.com/q/05badf52-27d8-4742-8532-91f442850ac7/ https://minutesof.com/q/05badf52-27d8-4742-8532-91f442850ac7/ “And the only two functioning switched scale up networks in the world today are the ones that power NVIDIA GPUs and Amazon's Trainiums.” — Gavin Baker, Sohn Conference Foundation Fri, 15 May 2026 18:17:41 +0000 inference https://minutesof.com/q/9f8eb50e-f98e-49c0-8051-f5a028e01a39/ https://minutesof.com/q/9f8eb50e-f98e-49c0-8051-f5a028e01a39/ “And so Trainium is for sure the most underestimated, not only because of those design choices, but because the all of these frontier models are what are called mixture of expert models.” — Gavin Baker, Sohn Conference Foundation Fri, 15 May 2026 18:17:41 +0000 inference https://minutesof.com/q/77488a06-75d9-432d-b0cd-0afa0e21f41f/ https://minutesof.com/q/77488a06-75d9-432d-b0cd-0afa0e21f41f/ “Tranium is going to be to 2026, especially in the second half of this year when Tranium three really ramps, as TPUs were to twenty twenty five.” — Gavin Baker, Sohn Conference Foundation Fri, 15 May 2026 18:17:41 +0000 inference https://minutesof.com/q/a91f962e-6711-454f-b1db-5c6d4437360d/ https://minutesof.com/q/a91f962e-6711-454f-b1db-5c6d4437360d/ “Tranium, by far. Tranium is going to be to 2026, especially in the second half of this year when Tranium three really ramps, as TPUs were to twenty twenty five.” — Gavin Baker, Sohn Conference Foundation Fri, 15 May 2026 18:17:41 +0000 inference https://minutesof.com/q/7c0eb714-2699-491f-b983-09b25778e0f7/ https://minutesof.com/q/7c0eb714-2699-491f-b983-09b25778e0f7/ “What happens when 5% of the world's population is using these models the way the cutting edge 10 basis points are? Like, it's just it's unimaginable. This is why orbital compute is a necessity.” — Gavin Baker, Sohn Conference Foundation Fri, 15 May 2026 18:17:41 +0000 inference https://minutesof.com/q/20c46b63-3663-4f9f-9324-17a82821dfcb/ https://minutesof.com/q/20c46b63-3663-4f9f-9324-17a82821dfcb/ “And we're in an insane shortage despite spending cumulatively trillions of dollars. What happens when 5% of the world's population is using these models the way the cutting edge 10 basis points are?” — Gavin Baker, Sohn Conference Foundation Fri, 15 May 2026 18:17:41 +0000 inference https://minutesof.com/q/eb16479a-ee2b-4d2c-9779-1e7eb5762367/ https://minutesof.com/q/eb16479a-ee2b-4d2c-9779-1e7eb5762367/ “We're just moving from these all you can eat plans to usage based plans with overage, where those usage tokens cost a lot more, and we're finding out that there's we're nowhere near the amount of, you know, people ceiling price for how much they'll spend.” — Gavin Baker, Sohn Conference Foundation Fri, 15 May 2026 18:17:41 +0000 inference https://minutesof.com/q/44c831c5-138e-45c3-8b54-9ad0278248ff/ https://minutesof.com/q/44c831c5-138e-45c3-8b54-9ad0278248ff/ “And then as the horizon of AI models gets longer, right, rather than a question answer immediately, the question answer becomes ten minutes, hours, days, the value of all the reasoning is gone.” — Dylan Patel, Swole as a Service Wed, 13 May 2026 20:00:36 +0000 inference https://minutesof.com/q/a9706075-6705-4eb4-ad0e-013b2f735206/ https://minutesof.com/q/a9706075-6705-4eb4-ad0e-013b2f735206/ “In the case of OpenAI, they don't show you the whole reasoning chain. In the case of Anthropic, they do.” — Dylan Patel, Swole as a Service Wed, 13 May 2026 20:00:36 +0000 inference https://minutesof.com/q/bf34d090-4f22-4978-bd08-307b1566b305/ https://minutesof.com/q/bf34d090-4f22-4978-bd08-307b1566b305/ “with software changing literally multiple times a week, right, PyTorch has nightlies, VLM has nightlies, Asteeling has nightlies, CUDA drivers update constantly. You just go You go through the whole list.” — Dylan Patel, TensorWave Thu, 30 Apr 2026 18:18:14 +0000 inference https://minutesof.com/q/7d67261b-caa3-4beb-8c61-2b0c52ec53da/ https://minutesof.com/q/7d67261b-caa3-4beb-8c61-2b0c52ec53da/ “And furthermore, with software changing literally multiple times a week, right, PyTorch has nightlies, VLM has nightlies, Asteeling has nightlies, CUDA drivers update constantly. You just go You go through the whole list.” — Dylan Patel, TensorWave Thu, 30 Apr 2026 18:18:14 +0000 inference https://minutesof.com/q/a8d7ee01-6bf0-4ca4-828e-77222cf7a242/ https://minutesof.com/q/a8d7ee01-6bf0-4ca4-828e-77222cf7a242/ “When someone has really good networking with these leading edge techniques to run a model such as KIMI or DeepSeq called disaggregated PD or wide EP, these are two different techniques, network performance is a humongous factor.” — Dylan Patel, Aria Networks Thu, 16 Apr 2026 02:27:54 +0000 inference https://minutesof.com/q/eae8cac0-9d21-4f54-97aa-ef5eae6fa378/ https://minutesof.com/q/eae8cac0-9d21-4f54-97aa-ef5eae6fa378/ “code agent revenue has gone from a couple billion to north of 10,000,000,000 in like a very short amount of time. And these, the horizon of these has also increased dramatically,” — Dylan Patel, Daytona Tue, 07 Apr 2026 17:12:37 +0000 inference https://minutesof.com/q/efa4c687-8b33-4fdc-88bc-eb9d63667ddd/ https://minutesof.com/q/efa4c687-8b33-4fdc-88bc-eb9d63667ddd/ “so it's grounded in the laws of physics, but able to predict 10,000 times faster.” — Jensen Huang, NVIDIA Thu, 05 Feb 2026 02:53:33 +0000 inference https://minutesof.com/q/eb1ea83d-8c91-4ef7-8dde-aedbbff7c2b4/ https://minutesof.com/q/eb1ea83d-8c91-4ef7-8dde-aedbbff7c2b4/ “PhysicsNEMO is essentially a physics aware AI model simulation system and AI framework that allows us to create these AI models that are either trained by principled simulators or work alongside principled simulators” — Jensen Huang, NVIDIA Thu, 05 Feb 2026 02:53:33 +0000 inference https://minutesof.com/q/4f81d78a-5e9e-4884-9e86-06863a537aee/ https://minutesof.com/q/4f81d78a-5e9e-4884-9e86-06863a537aee/ “AI reduces the cost of intelligence or create the abundance of intelligence by orders of magnitude.” — Jensen Huang, Cisco Wed, 04 Feb 2026 04:34:47 +0000 inference https://minutesof.com/q/d63342d7-794d-43b2-8529-800ca943aa5a/ https://minutesof.com/q/d63342d7-794d-43b2-8529-800ca943aa5a/ “for a lot of people, I'm fine to spend 10x the price on something that completes 10x faster. So Cerberus sort of just makes a ton of sense there.” — Dylan Patel, TBPN Tue, 03 Feb 2026 23:08:03 +0000 inference https://minutesof.com/q/b5b15eca-a046-4a24-8309-0cfa9d92ceec/ https://minutesof.com/q/b5b15eca-a046-4a24-8309-0cfa9d92ceec/ “One example is one that DeepSeek open sourced over December of last yearJanuary, February of this year, where a single replica of inference for a single model is going to be like 140 GPUs.” — Dylan Patel, Clockwork Fri, 21 Nov 2025 17:18:51 +0000 inference https://minutesof.com/q/e37df648-a474-40f3-9e9c-b531f05a918e/ https://minutesof.com/q/e37df648-a474-40f3-9e9c-b531f05a918e/ “what sort of NVIDIA's pitching is like a split of inference into two workloads, and we'll see if they're successful. There's a lot of challenges on the infrastructure side.” — Dylan Patel, Together AI Fri, 03 Oct 2025 20:23:30 +0000 inference https://minutesof.com/q/8896a5c5-5fee-4e45-86fb-6379c1c3eec2/ https://minutesof.com/q/8896a5c5-5fee-4e45-86fb-6379c1c3eec2/ “NVIDIA's next generation actually has something very different. They're not saying, Hey, there's a training GPU and an inference GPU, right? Because either is fine.” — Dylan Patel, Together AI Fri, 03 Oct 2025 20:23:30 +0000 inference https://minutesof.com/q/b34c2dfe-463a-4923-8ba7-41d8b89b38a6/ https://minutesof.com/q/b34c2dfe-463a-4923-8ba7-41d8b89b38a6/ “And so you've sort of Which is the exact opposite of what you would have expected. Oh, use the big thing for training and use the small thing for inference.” — Dylan Patel, Together AI Fri, 03 Oct 2025 20:23:30 +0000 inference https://minutesof.com/q/63137c12-d683-47eb-9bf7-16977addf85c/ https://minutesof.com/q/63137c12-d683-47eb-9bf7-16977addf85c/ “OpenAI has said they're running production inference on GV200 a couple of months ago, in fact. Right?” — Dylan Patel, Together AI Fri, 03 Oct 2025 20:23:30 +0000 inference https://minutesof.com/q/62b39bc8-1e44-4f9d-b9b5-d25bb37cda66/ https://minutesof.com/q/62b39bc8-1e44-4f9d-b9b5-d25bb37cda66/ “AWS has been decelerating revenue. Year on year revenue has been falling consistently. And and our big call is that it's actually going to start reaccelerating.” — Dylan Patel, a16z Mon, 22 Sep 2025 13:01:51 +0000 inference https://minutesof.com/q/c5f66780-d7e1-4b21-9d13-444df206cd01/ https://minutesof.com/q/c5f66780-d7e1-4b21-9d13-444df206cd01/ “Today, it's 980,000,000,000,000 tokens. So from 9,000,000,000,000 to nine eighty, it's a 100 x increase in a year.” — Brad Gerstner, CNBC Television Thu, 28 Aug 2025 17:12:24 +0000 inference https://minutesof.com/q/9e102380-dbd9-4dbb-acd8-a43420fa4c7f/ https://minutesof.com/q/9e102380-dbd9-4dbb-acd8-a43420fa4c7f/ “A large part of export controls, if they work, is just that the amount of AI that can be run-in China is going to be much lower.” — Dylan Patel, Lex Fridman Mon, 03 Feb 2025 00:12:13 +0000 inference https://minutesof.com/q/5f490d92-e1c8-4877-b369-6b470be69d65/ https://minutesof.com/q/5f490d92-e1c8-4877-b369-6b470be69d65/ “And then we started scaling around inference time compute. And it's very clear that we have now added a third axis of scaling performance, and that is reasoning.” — Gavin Baker, All-In Podcast Sat, 04 Jan 2025 00:23:00 +0000 inference https://minutesof.com/q/b30df38d-a975-4d8d-9f97-4b9eeeed3b27/ https://minutesof.com/q/b30df38d-a975-4d8d-9f97-4b9eeeed3b27/ “inference compute is gonna be the the kind of derivative winner of that. We're gonna run out of GPUs, accelerators, compute in 2025 the same way we did in '23.” — Gavin Baker, All-In Podcast Sat, 04 Jan 2025 00:23:00 +0000 inference https://minutesof.com/q/b30df38d-a975-4d8d-9f97-4b9eeeed3b27/ https://minutesof.com/q/b30df38d-a975-4d8d-9f97-4b9eeeed3b27/ “inference compute is gonna be the the kind of derivative winner of that. We're gonna run out of GPUs, accelerators, compute in 2025 the same way we did in '23.” — Gavin Baker, All-In Podcast Sat, 04 Jan 2025 00:23:00 +0000 inference https://minutesof.com/q/3af95131-05a1-45d4-ae2d-9578bd46190d/ https://minutesof.com/q/3af95131-05a1-45d4-ae2d-9578bd46190d/ “Cost increase for a single token to be generated is four to five x, but then I'm generating 10 x as many tokens.” — Bill Gurley, Bg2 Pod Mon, 23 Dec 2024 20:25:20 +0000 inference https://minutesof.com/q/9aed22e0-c898-4e8e-9a74-966679c754b1/ https://minutesof.com/q/9aed22e0-c898-4e8e-9a74-966679c754b1/ “I outputted a thousand tokens to I outputted 11,000 tokens. I've 10x'd my spend to generate no. Not the same thing. Right? It's higher quality.” — Bill Gurley, Bg2 Pod Mon, 23 Dec 2024 20:25:20 +0000 inference https://minutesof.com/q/476ea39f-bfb6-495d-9a59-81ea69699b0d/ https://minutesof.com/q/476ea39f-bfb6-495d-9a59-81ea69699b0d/ “It generates tons of things. It's like it it sometimes switches between Chinese and English. Right? Like, whatever it is. It's thinking. Right? It's churning.” — Bill Gurley, Bg2 Pod Mon, 23 Dec 2024 20:25:20 +0000 inference https://minutesof.com/q/1524c47b-e0e1-402a-b743-111da2d0e959/ https://minutesof.com/q/1524c47b-e0e1-402a-b743-111da2d0e959/ “It generates tons of things. It's like it it sometimes switches between Chinese and English. Right? Like, whatever it is. It's thinking. Right?” — Dylan Patel, BG2 Pod Mon, 23 Dec 2024 20:12:56 +0000 inference https://minutesof.com/q/168ba19f-f3d8-48fd-bbbd-632f2cb46217/ https://minutesof.com/q/168ba19f-f3d8-48fd-bbbd-632f2cb46217/ “There was a podcast recently where they kind of flipped everything on their head and they said, well, if we're not doing that anymore, it's way better because we can just move on to inference, which is getting cheaper and you won't have to spend all this CapEx.” — Bill Gurley, Bg2 Pod Thu, 12 Dec 2024 18:20:19 +0000 inference https://minutesof.com/q/b7e7fd8f-2e86-4705-b53e-c249725aab84/ https://minutesof.com/q/b7e7fd8f-2e86-4705-b53e-c249725aab84/ “But if you batch higher, k b cache is not just a memory capacity issue, it's also a memory bandwidth issue.” — Dylan Patel, Scaling Intelligence Tue, 12 Nov 2024 04:17:00 +0000 inference https://minutesof.com/q/2548d082-f15b-4fca-b6d4-4cfc177ee7e5/ https://minutesof.com/q/2548d082-f15b-4fca-b6d4-4cfc177ee7e5/ “If you ask it to, like, generate a web scraper, in the standard one it'll just start outputting code, and it'll be maybe like a four k sequence length.” — Dylan Patel, Scaling Intelligence Tue, 12 Nov 2024 04:17:00 +0000 inference https://minutesof.com/q/50e70e24-1d30-4bef-bf58-58b1d0263aa5/ https://minutesof.com/q/50e70e24-1d30-4bef-bf58-58b1d0263aa5/ “he said as a consequence of that, inference is going to a 100 x, thousand x, a million x, maybe even a billion x.” — Brad Gerstner, BG2 Pod Sun, 13 Oct 2024 10:35:00 +0000 inference https://minutesof.com/q/27a00324-83ae-472e-8e7b-d4900e14122e/ https://minutesof.com/q/27a00324-83ae-472e-8e7b-d4900e14122e/ “in order to get linear improvement, you have to do maybe 10X the amount of processing. And this is all inference. So what are the implications of that?” — Bill Gurley, BG2 Pod Wed, 25 Sep 2024 16:21:38 +0000 inference https://minutesof.com/q/a913c721-311e-4f5d-a524-73f6cda49ae1/ https://minutesof.com/q/a913c721-311e-4f5d-a524-73f6cda49ae1/ “I think they're going to go really fast to GPT-4.5 compute, which means you're going to get, using these orders of magnitude, you're going get a 100x improvement really fast.” — Gavin Baker, Invest Like the Best Tue, 27 Aug 2024 08:00:00 +0000 inference https://minutesof.com/q/1c159533-7002-43df-a5db-be3d8cc9d824/ https://minutesof.com/q/1c159533-7002-43df-a5db-be3d8cc9d824/ “and it's been very well established in multiple papers from both Google and Microsoft research that for every order of magnitude increase in the data you use to train an algorithm, the quality of the AI doubles.” — Gavin Baker, Invest Like the Best Tue, 26 Nov 2019 10:30:00 +0000 inference https://minutesof.com/q/32e64b43-db5d-48ee-815c-272156cf2971/ https://minutesof.com/q/32e64b43-db5d-48ee-815c-272156cf2971/ “The single most predictive element of knowledge about AI quality is the quantity of data used to train the algorithm.” — Gavin Baker, Invest Like the Best Tue, 26 Nov 2019 10:30:00 +0000 inference https://minutesof.com/q/c6581532-cbbe-425f-bb38-995439fb7bb4/ https://minutesof.com/q/c6581532-cbbe-425f-bb38-995439fb7bb4/ “The only thing that has enabled the AI revolution that we're living through, which I think we're at the bottom of the first inning in, is one, we had the ability to do cloud computing, so just apply significantly more computational power to old algorithms, and then b, we had dramatically more data.” — Gavin Baker, Invest Like the Best Tue, 26 Nov 2019 10:30:00 +0000 inference