Gerstner says Microsoft, Amazon, and Meta all signaled massive inference and infrastructure investments driving NVIDIA higher.
“It's up because Amazon told us the same thing. It's up because Susan Lee from Meta just told us yesterday that they're going to make huge future investments in infrastructure, read NVIDIA,”
Gerstner reports Jensen predicts inference will scale 100x to billion-x with 40% of NVIDIA revenue already from inference
“he said as a consequence of that, inference is going to a 100 x, thousand x, a million x, maybe even a billion x.”
Gerstner says inference costs fell 90% last year and will drop another 90% in coming months.
“What Sunny has told us and other people in the field have told us that inference is going to drop by another 90% over the cost of the next period of months.”
Gerstner states inference time reasoning requires way more compute than scaling pretraining.
“inference time reasoning is way more compute intensive than simply scaling pretraining. Why don't you walk us through? We have a really interesting graph here about why that's the case”
Gerstner says Google's monthly token generation exploded 100x in a year, from 9 trillion to 980 trillion tokens.
“Today, it's 980,000,000,000,000 tokens. So from 9,000,000,000,000 to nine eighty, it's a 100 x increase in a year.”
Gerstner says the world will be supply constrained on inference compute indefinitely, citing NVIDIA's Grok acquisition.
“The world is going to be supply constrained on inference compute as far as the eye can see. It was part of the inspiration. We were also investors in Grok.”
Gerstner argues that token production and consumption is the fundamental basis of all AI intelligence.
“There is no intelligence. There's no consumer chat GPT. There's no enterprise intelligence. There's no clawed code without the production of tokens.”