I have seen a similar curve in software development - 18 months ago it was highly debatable whether application of AI tools was a net-positive or not. 12 months ago the rate of improvement was very clear and the net-positive wasn't much of a question anymore. Today, the net positive contributions of LLM agents to software development and maintenance are undeniable, and as you say - there seems to be some plateauing, maybe - it's hard to tell.

Where I think things are going to continue to "improve" at a slower pace is application of the LLM agents to do better software development and maintenance. I've seen more serious Linux kernel bugs discovered - and patched - in the past 6 months than in the previous 5 years. That's a shift in the landscape, the older OSs aren't as secure for as long as they used to be, but once we're through this transition the newer OS versions should be significantly more secure than before - getting all the old stuff updated is going to be a painful transition.

Some of the continuing improvement is in using LLM agents to build better software building tools - as those come online and start to shed their sloppiness, they should genuinely accelerate a lot of aspects of software development. They'll never accelerate the process of extracting from users just what it is they need, but the turn around from having a product idea to having a product to try continues to shorten - as it has been shortening for 60+ years - but LLM agents have resulted in a significant speedup compared to previous progress.

I’m not worried about a rogue AI unleashing some bioweapon or launching a nuclear attack.

(In Yoda delivery) You should be. We all should be, not because AI will do it autonomously, or in retrospect unexpectedly, but because careless people may let this happen through AI where we have prevented it from happening up to this point.