Overheard an AI researcher say 'we're running out of clean training data' and it hit me different

I was at a coffee shop downtown two weeks ago and this guy on a Zoom call said they've already scraped most of the internet... like, the good parts are gone. He said new models are starting to train on AI-generated text and it's making them dumber over time. Made me wonder if all these AI tools are gonna plateau hard in the next year or two. Has anyone else heard about data quality dropping off like this?

2 comments

2 Comments

colemiller1mo ago

Start filtering training data manually. Garbage in, garbage out.

emma_hart1mo ago

Training on AI slop to make more AI slop, what could go wrong? Feels like we're just gonna end up with a snake eating its own tail until the whole thing collapses. Ngl, it's kind of funny watching them panic over running out of actual human thoughts to copy. They spent years scraping everyone's tweets and blog posts for free, and now they're stuck feeding each other's boring homework back and forth. Maybe if they had to actually learn something original instead of stealing it, they'd have a problem. Good luck convincing investors that their golden goose is slowly turning into a pigeon.