Patel explains Bumpgate issue where thermal expansion differences caused GPU solder ball failures in Apple laptops.
“And what ended up happening is because of that different rate of expansion, the solder balls connecting the chip and the board would crack.”
Patel explains that Apple's rushed model has empty experts not receiving tokens due to flawed sparse MOE routing.
“Basically in between every layer, the router can route to whatever expert it wants to, And it learns which expert to route to. And and each expert learns its own independent things”
Patel describes Apple's model having empty experts that never get routed to, indicating wasted training capacity.
“if you actually look at the model, oftentimes won't even route tokens to certain experts. So it was like a waste of training.”
Patel describes the Bumpgate incident where NVIDIA laptop GPUs had defective solder balls.
“There's a generation of NVIDIA GPUs for laptops. Right? And and chips have solder balls on the bottom. Right?”
Patel reports memory prices tripled from $3-4 per gigabyte to $12, adding $100 iPhone cost.
“Each gig cost used to cost roughly 3 or $4, so it's $50. But now the price of memory is like tripled. Let's call it if it's now it's $12 per gig for DDR.”
Patel says TSMC is forcing Apple, Qualcomm, and MediaTek off three-nanometer, pushing them toward Intel.
“They're telling Qualcomm and MediaTek to get off and all three of these companies are like, oh, maybe we should use Intel because they're not telling us to get off”