Patel explains agentic training only needs to sync tokens every few minutes versus weights every seconds in pre-training.
“When you're doing these rollouts and especially as things get more and more agentic and training, you might not only need to send not the entire weights but just the tokens that are relevant.”