Large language models cannot retain new information told to them during conversations.
“Current large models lack continuous learning. You can’t tell them something and expect them to remember.”
Karpathy proposes a small model of one to two billion parameters combined with external memory that can grow over time.
“ (which he estimates needs only one or two billion parameters) paired with a structured external memory system capable of self-reinforcing growth. Following this idea, he released a model called ”