Patel asserts data quality is the primary determinant of model quality, not just compute or architecture.
“the data processing, data filtering, data quality is the number one determinant of the model quality, and then a lot of the training code is the determinant on how long it takes to train and how fast your experimentation is.”
Patel notes YouTube holds unique video data that gives Google training advantage despite scraping restrictions.
“I don't know what ungodly percentage of video is only on available on YouTube. And sure, you can scrape YouTube, blah blah blah.”