Karpathy shares browser-playable source for his Lord of the Rings test and jokes about GTA Hobbiton
“More on the pelican on the bicycle test from @simonw: https://t.co/OXmtODyTKj I uploaded the source here so it's playable in the browser, forkable etc. https://t.co/w3Nctc888d Look out for GTA Hobbiton dropping before GTA VI :)”
Karpathy tests Opus 5 with first paragraph of Lord of the Rings and 1M token budget at roughly $10 cost
“We're starting to leave the territory where you'd test an LLM by e.g. "create an svg of pelican on a bicycle". As one idea to generalize it, I was interested what Opus 5 would do if I gave it the first paragraph of the Lord of the Rings, a 1M token budget (~$10) and asked for https://t.co/ybIpXhYSsj”
Karpathy compares human sleep to a distillation process that consolidates daily experiences into long-term memory, a capability he says LLMs lack since they restart with empty context windows.
“I feel like when I’m awake, I’m building up a context window of stuff that’s happening during the day. But when I go to sleep, something magical happens… a process of distillation into the weights of my brain. We don’t have an equivalent of that in LLMs. When you boot them up, they have zero tokens in the window. They’re always restarting from scratch.”
Current large language models cannot retain information told to them across sessions.
“have no continual learning. You can't tell it one thing and expect it to remember.”
Karpathy finds large language models useful for building personal knowledge bases on research topics.
“Something I’m finding very useful recently: using LLMs to build personal knowledge bases for various topics of research interest.”