Testing language models is moving beyond simple creative tasks like generating SVG images of bicycling pelicans.
“We’re starting to leave the territory where you’d test an LLM by e.g. ‘create an svg of pelican on a bicycle,’”
Karpathy says the era of testing language models with simple prompts is ending.
“We're starting to leave the territory where you'd test an LLM by e.g. ”