Patel reports reasoning models improved college math test scores from 4% to 80% in six months.
“Basically, our college level math tests were initially 4% for before this innovation was happened to now they're scoring 80%, the best leading edge models. Right? And this has just happened in six months.”
Patel illustrates that language models historically failed at basic math comparisons without reasoning capabilities.
“And it would say, yes. It's bigger. Even though, like, everyone knows that nine dot one one is is way smaller than nine dot nine.”