Baker says ARC AGI scores jumped from 8% to 95% in three months with reasoning models.
“The reason we've had all this progress, maybe we could show like the ARC AGI slide where you had zero to eight over four years, zero to 8% intelligence, and then you went from 8% to 95% in three months when the first reasoning model came out from OpenAI.”
Baker proposes a new metric decomposing MFU that he developed the morning of the interview.
“MFU is the most important metric because it gives you all of these advantages and ways to differentiate yourself amongst five people, six people who've trained these GPT core class models.”