We really should be slowing down and looking at the effects of AI. It has too much potential to fuck everything up to not think about it a bit, especially if recursive self-improvement is on the horizon.

This is Anthropic's entire marketing strategy: get people thinking that LLMs can do more than they actually can through fear. This is what they constantly do and have done for at least the last two years. They also tend to stretch or bury the truth in their "studies" with these goals in mind.

Fear, especially of the unknown, is very powerful. If you fear it the way they want you to, that feeling will convince you that the LLMs are going to take over any day now, and once enough people think that, it pumps Anthropic's valuation.

I do agree that we should slow down, but not necessarily because of any recursion (they claim they can already do this with "agents", another term they stole to make LLMs seem more capable). I think we should stop because it's a huge bubble, and the more we pump into it the louder the pop will be. Not only that, but is a monumental waste of resouces, all to generate half-baked slop most of the time.