Patel argues interconnects between servers are now the key bottleneck, not individual chip speed.
“The interconnects is the biggest bottleneck right? And so the problem set is it moved from you know just one, you know, hey, my chip is faster.”
Patel explains NVLink connects only 72 GPUs while Google ICI connects 8,000 chips without switches, creating architectural trade-offs.
“NVIDIA, the NVLink can only connect 72 GPUs. For Google, their ICI can connect 8,000 chips at super high bandwidth, but you have to pass through other chips to get there because there's no switch.”