Huang says models may run 10,000 internal inferences before answering, including tree search and reinforcement learning.
“The idea that a model before it answers your answer had already done internal inference 10,000 times, it's probably not unreasonable. And it's probably done tree search, it's probably done reinforcement learning on that,”
Huang contrasts Moore's law (100x per decade) with current AI acceleration (1 million times per decade).
“Remember, Moore's law was two times every 18, 10 times every five years, a 100 times every ten. Okay? But where are we now? A million times every ten years.”
Huang says NVIDIA will quadruple Coherent's indium phosphide fab capacity in one year versus fifty years for original.
“We're gonna take almost one year to build a four x. Are you guys the scale up is incredible. That tells you about the incredible demand that we have for accelerated computing,”