Who is on this subjectordered by the date of their first quote here
1 of 2 lane rests on fewer than 5 quotes and is marked thin. Offsets are days from the middle first-quote date, 20 Jun 2020 — a date, and nothing else. It is not a claim about who reached a view first.
-
David Friedberg
3
thin
20 Jun 2020 – 3 Oct 2025
+0 d
- Dylan Patel 12 9 Aug 2023 – 25 Aug 2026 +1145 d
The chronologysourced and dated, oldest first
-
-
David Friedberg
Friedberg advocates ending qualified immunity and retraining police as a service rather than military force.
“I I'm a huge fan of ending qualified immunity. I think that doesn't make any sense. I think we have to stop arming our police like their military.”
-
-
-
Dylan Patel
Patel predicts five to seven companies will train GPT-4 scale models in the next year.
“I do believe that there's going to be five to seven companies that will have a GPT-four size model, at least, right, in terms of total flops.”
-
-
-
Dylan Patel
Patel explains that AdamW optimizer requires four bytes per parameter, creating 400 gigabytes of data for 100B parameter models.
“when you look at LLMs, people use AdamW. And AdamW as an optimizer is like four bytes per parameter roughly, if I recall correctly the optimizer state.”
-
Dylan Patel
Patel calculates that multi-trillion parameter models require transmitting 40 terabytes of data every two seconds during training.
“They're doing it for like multi trillion. Right? So let's call it 10,000,000,000,000 parameters, four bytes parameter, that's 40 terabytes of data you need to transmit in two seconds.”
-
-
-
Dylan Patel
Patel explains DeepSeek v3 base is trained once, then post-trained differently to create chat versus reasoning models.
“This reasoning model has a lot of overlapping training steps to DeepSeek v three, and it's confusing that you have a base model called v three that you do something to to get a chat model, and then you do some different things to get a reasoning model.”
-
Dylan Patel
Patel says current models use 100,000 GPUs while next generation will require hundreds of thousands or millions.
“And next generation models that are trained on hundreds of thousands or even millions GPUs, right?”
-
Dylan Patel
Patel says GPUs in a 16,000 GPU cluster show up to 10% speed variation from manufacturing differences.
“But even within a cluster of like 16 k GPUs, GPUs will be slower by up to 10%. Right? So there there's a lot of variation even in the manufacturing.”
-
Dylan Patel
Patel explains training a 70 billion parameter model requires passing four times that data every training step without overlapping communication.
“So if you're training like a 70,000,000,000 parameter model, you need to pass four x that number every single training step. And when you're passing that, you can't really overlap communications and compute.”
-
Dylan Patel
Patel reports that OpenAI's Orion training run did not improve enough over GPT-4.5 to qualify as GPT-5.
“There were hopes that Orion could be used for for GPT five, but its improvement was, like, not enough to be, like, really a GPT five.”
-
Dylan Patel
Patel says GPT-5 will combine massive pre-training like GPT-4.5 with massive post-training like o1 and o3.
“GPT five, as Sam calls it, is is gonna be a model that has huge pre training scale, right, like GPT 4.5, but also huge post training scale like o one and o three and continuing to scale that up.”
-
David Friedberg
Friedberg credits a 24-day intensive Jay Robinson wrestling camp at University of Minnesota for teaching him positive affirmations.
“It was, like, a twenty four day intensive wrestling camp in Minnesota. I forgot how many days specifically, but it was a Jay Robinson wrestling camp at the University of Minnesota.”
-
David Friedberg
Friedberg attended a 24-day intensive Jay Robinson wrestling camp at University of Minnesota where he learned positive affirmations.
“It was like a twenty four day intensive wrestling camp in Minnesota. I forgot how many days specifically, but it was a Jay Robinson wrestling camp at the University of Minnesota.”
-
Dylan Patel
Patel says B200 is better for training while GB200 is better for inference, reversing expected use.
“And so you've sort of Which is the exact opposite of what you would have expected. Oh, use the big thing for training and use the small thing for inference.”
-
-
-
Dylan Patel
Patel explains agentic training only needs to sync tokens every few minutes versus weights every seconds in pre-training.
“When you're doing these rollouts and especially as things get more and more agentic and training, you might not only need to send not the entire weights but just the tokens that are relevant.”
-
Dylan Patel
Patel argues labs will allocate less compute to inference over time, contrary to consensus belief.
“So, this is sort of what I personally believe that the labs are going to allocate less and less compute to inference over time, I think is very non consensus, right?”
-