https://minutesof.com/on/training/ Everyone on the record about training: 15 verbatim quotes from 2 people, June 2020 to August 2026, in one chronology, each with a timestamp and a link to… en Sun, 30 Aug 2026 22:33:18 +0000 https://minutesof.com/q/469a18e6-8605-46bd-bea0-fa21bd5502b3/ https://minutesof.com/q/469a18e6-8605-46bd-bea0-fa21bd5502b3/ “So, this is sort of what I personally believe that the labs are going to allocate less and less compute to inference over time, I think is very non consensus, right?” — Dylan Patel, Dwarkesh Patel Tue, 25 Aug 2026 15:57:53 +0000 training https://minutesof.com/q/86748cbb-055a-4578-92be-361a647d16ac/ https://minutesof.com/q/86748cbb-055a-4578-92be-361a647d16ac/ “When you're doing these rollouts and especially as things get more and more agentic and training, you might not only need to send not the entire weights but just the tokens that are relevant.” — Dylan Patel, TBPN Tue, 03 Feb 2026 23:08:03 +0000 training https://minutesof.com/q/b34c2dfe-463a-4923-8ba7-41d8b89b38a6/ https://minutesof.com/q/b34c2dfe-463a-4923-8ba7-41d8b89b38a6/ “And so you've sort of Which is the exact opposite of what you would have expected. Oh, use the big thing for training and use the small thing for inference.” — Dylan Patel, Together AI Fri, 03 Oct 2025 20:23:30 +0000 training https://minutesof.com/q/a52409fd-1020-4c4e-98c8-069c694790f2/ https://minutesof.com/q/a52409fd-1020-4c4e-98c8-069c694790f2/ “It was like a twenty four day intensive wrestling camp in Minnesota. I forgot how many days specifically, but it was a Jay Robinson wrestling camp at the University of Minnesota.” — David Friedberg, NPTE Final Frontier Fri, 03 Oct 2025 11:51:33 +0000 training https://minutesof.com/q/6877f97a-1da9-45a3-8085-2cb2668ed870/ https://minutesof.com/q/6877f97a-1da9-45a3-8085-2cb2668ed870/ “It was, like, a twenty four day intensive wrestling camp in Minnesota. I forgot how many days specifically, but it was a Jay Robinson wrestling camp at the University of Minnesota.” — David Friedberg, NPTE Final Frontier Thu, 02 Oct 2025 02:28:47 +0000 training https://minutesof.com/q/808fcaf5-9216-420e-b11b-f8a91aeadae8/ https://minutesof.com/q/808fcaf5-9216-420e-b11b-f8a91aeadae8/ “GPT five, as Sam calls it, is is gonna be a model that has huge pre training scale, right, like GPT 4.5, but also huge post training scale like o one and o three and continuing to scale that up.” — Dylan Patel, Alex Kantrowitz Wed, 23 Apr 2025 16:30:06 +0000 training https://minutesof.com/q/8373ef31-935a-4e8d-90e3-7c6565db225c/ https://minutesof.com/q/8373ef31-935a-4e8d-90e3-7c6565db225c/ “There were hopes that Orion could be used for for GPT five, but its improvement was, like, not enough to be, like, really a GPT five.” — Dylan Patel, Alex Kantrowitz Wed, 23 Apr 2025 16:30:06 +0000 training https://minutesof.com/q/2bc44e6d-a415-482b-bcb0-f9c59a1b4198/ https://minutesof.com/q/2bc44e6d-a415-482b-bcb0-f9c59a1b4198/ “So if you're training like a 70,000,000,000 parameter model, you need to pass four x that number every single training step. And when you're passing that, you can't really overlap communications and compute.” — Dylan Patel, Prime Intellect AI Tue, 25 Mar 2025 18:21:45 +0000 training https://minutesof.com/q/e4cb87fa-869e-4b13-8b85-bd4d3fb6cb18/ https://minutesof.com/q/e4cb87fa-869e-4b13-8b85-bd4d3fb6cb18/ “But even within a cluster of like 16 k GPUs, GPUs will be slower by up to 10%. Right? So there there's a lot of variation even in the manufacturing.” — Dylan Patel, Prime Intellect AI Tue, 25 Mar 2025 18:21:45 +0000 training https://minutesof.com/q/561bd369-2c88-4ea1-8fa4-3d02389be50d/ https://minutesof.com/q/561bd369-2c88-4ea1-8fa4-3d02389be50d/ “And next generation models that are trained on hundreds of thousands or even millions GPUs, right?” — Dylan Patel, Special Competitive Studies Project Thu, 13 Mar 2025 16:42:15 +0000 training https://minutesof.com/q/02fd4a40-e301-43f7-be83-33801b8010a9/ https://minutesof.com/q/02fd4a40-e301-43f7-be83-33801b8010a9/ “This reasoning model has a lot of overlapping training steps to DeepSeek v three, and it's confusing that you have a base model called v three that you do something to to get a chat model, and then you do some different things to get a reasoning model.” — Dylan Patel, Lex Fridman Mon, 03 Feb 2025 00:12:13 +0000 training https://minutesof.com/q/2b1dd77d-4b29-490c-a1f9-fd32caff427b/ https://minutesof.com/q/2b1dd77d-4b29-490c-a1f9-fd32caff427b/ “They're doing it for like multi trillion. Right? So let's call it 10,000,000,000,000 parameters, four bytes parameter, that's 40 terabytes of data you need to transmit in two seconds.” — Dylan Patel, Scaling Intelligence Tue, 12 Nov 2024 04:17:00 +0000 training https://minutesof.com/q/9b0a4eda-79e5-4213-b0df-76f45b715526/ https://minutesof.com/q/9b0a4eda-79e5-4213-b0df-76f45b715526/ “when you look at LLMs, people use AdamW. And AdamW as an optimizer is like four bytes per parameter roughly, if I recall correctly the optimizer state.” — Dylan Patel, Scaling Intelligence Tue, 12 Nov 2024 04:17:00 +0000 training https://minutesof.com/q/fa2d7b01-051f-42f8-ab72-c441507a20aa/ https://minutesof.com/q/fa2d7b01-051f-42f8-ab72-c441507a20aa/ “I do believe that there's going to be five to seven companies that will have a GPT-four size model, at least, right, in terms of total flops.” — Dylan Patel, The Inside View Wed, 09 Aug 2023 15:00:23 +0000 training https://minutesof.com/q/c100e8e1-efd4-420f-aa63-79335d6d7771/ https://minutesof.com/q/c100e8e1-efd4-420f-aa63-79335d6d7771/ “I I'm a huge fan of ending qualified immunity. I think that doesn't make any sense. I think we have to stop arming our police like their military.” — David Friedberg, All-In Podcast Sat, 20 Jun 2020 01:15:25 +0000 training