Patel explains long context windows fail because pre-training uses short context and lacks data for longer ranges.
“But it's like, what data do I have? That's why my two fifty ks to one mil context is trash anyways, because there's no data on this stuff.”
Patel says datacenter model hard to justify selling to individuals, only B2B makes sense for effort required
“@MichaelSte5069 @SemiAnalysis_ You’re gonna love our next newsletter :) Unfortunately it’s hard to justify selling it to individuals, B2B is the only way we can justify the immense effort we have in it”
Patel says SemiAnalysis has more China compute mapped than cited estimate
“@RothRottweiler @aqib_zakaria @hamandcheese @SemiAnalysis_ Idk about all that chief. We have more compute for China mapped than this”
Patel says SemiAnalysis tracks changes as they happen, with majority revenue from industry not hedge funds
“@damnang2 Our role is to track what's happening and provide updates as changes happen. Majority of my revenue is industry not hedge funds. Theyre the ones making actual useful decisions with the data. All of these items we reported were true and we explained them. The one caveat is with”
Patel claims evidence exists of organizations astroturfing false datacenter narratives with non-domestic funding
“@suchenzang There's a lot of evidence of organizations astro turfing false narratives about datacenter water use and noise, having funding from non domestic sources. Not sure what he accusing me of tho? I always tweet what I believe Surely you're not saying I have some alterior motive”
Patel asks Douglas if he can use the model without privacy protections
“@_sholtodouglas Can I use your model with none of my privacy.”
Patel says OpenAI explicitly stated at Hot Chips they bet against disaggregation
“@GavinSBaker @maxkan It is for KVs but they also just explicitly said they bet against Disagg at Hot Chips”