Gurley explains Strawberry requires 10X processing for linear improvement, meaning 10-100X inference costs for better solutions.
“in order to get linear improvement, you have to do maybe 10X the amount of processing. And this is all inference. So what are the implications of that?”
Gurley explains Strawberry's logarithmic scaling means 10X to 100X inference costs for linear improvement requiring selective application.
“The implication being in order to get linear improvement, you have to do maybe 10X the amount of processing. And this is all inference.”
Gurley questions hardware implications if LLM differentiation shifts from pre-training to inference focus.
“if the world shifts to inference sooner or faster, and if the path of differentiation for the language model companies becomes more inference based than pre training based, what does that mean for the hardware systems that people are using?”