Where if you feed AI input in you only get garbage out and it just eats itself.

"Mad cow disease" would be an apt description - this is how BSE originated:

BSE is thought to occur due to an infection by a misfolded protein, known as a prion.[3][6] Cattle are believed to have been infected by being fed meat-and-bone meal that contained either the remains of cattle who spontaneously developed the disease or scrapie-infected sheep products.[3][7] The United Kingdom was afflicted with an outbreak of BSE and vCJD in the 1980s and 1990s. The outbreak increased throughout the UK due to the practice of feeding meat-and-bone meal to young calves of dairy cows.

At the same time they are also pulling towards it because of the potential recursive self-improvement has. It's not about just the training data anymore, it's AI finding new training techniques, new kernel architecture and so on. It's a separate, potential scaling vector as the training data vector has pretty much been maxed out in terms of big benefits.

😏 Excuse me, sir, did you drop this emoticon?

Nope

Think of it like making a xerox copy of a xerox copy. The copy of the copy is always shittier.

Using synthetic data can escalate model collapse, as a model is only as good as its training data, which is partly why these LLM models "hallucinate", having been trained on a wealth of garbage from Reddit.

They'll soon get special permission from the government to break into your house and scan your diary and grocery lists.

To me it felt more like humor, they over-used it intentionally to highlight the absurdity.