mixture of experts

Saidverbatim, newest first

1. Patel states mixture of experts architecture can achieve the same model performance with 30% less compute.

   “So you can get effectively the same performance model and evaluation scores with numbers like 30% less compute.”

   29:48 · Lex Fridman · 3 Feb 2025 · permalink