Patel says SemiAnalysis analyzed over $5 million worth of Claude production traces to benchmark agentic workloads.
“Initially, when we were benchmarking the difference between these chips and different engines, different schemes for parallelism, we were just running it, you know, fixed context length.”
Patel describes agentic workflows using 30,000-100,000 input tokens but generating only 1,000 output tokens.
“And and that makes, you know, initially, that'd be like, okay, well now I need a ton ton of compute to calculate all the prefilled tokens.”