Yes, precisely. It's not AI we need to be afraid of, but rather the same old capitalists that now have one more tool to ride the world into the ground in the name of greed.

This is exactly how I feel. I’m not even opposed to automation of these tasks on a fundamental level like some people might be. I just fear the transition won’t be thought out at all and will be devastating for the working class. The last time we had sudden automation of this scale was the Industrial Revolution which completely reshaped western society and was very bad for those who lived through it despite the eventual high modern standard of living it produced. I think this may end up going similarly.

This feels like the kind of narrative that OP is asking about. Everything that you've said sounds kind of sane and plausible, but it's not really borne out by the reality.

I think a realistic case for that is that the AI tools that come out in the next few years will be good enough to do a fair amount of white-collar work

That's just not what I've seen. I'd love to be able to automate the work of my white-collar team, but everything that we have seen thus far just seems incapable of providing any significant improvement. Software providers in our industry have implemented some gen AI tools, and even the sales reps for those providers are very wary of them. They've been able to reliably automate some elements of jobs, but it's an incremental improvement over the previous automation and it didn't consume a significant amount of time anyway.

There's absolutely no evidence that gen AI will have a significant impact on my white-collar industry. In fact, the evidence presently available indicates that this generation of AI does not have the capability to impact our industry.

other types of AI are already enabling robotic automation of factories and in the near-ish future might make self-driving truck fleets practical.

This is really the same story. Factories have been being automated since the industrial revolution. Self driving tech has been a year away for the last 10 years.

All of that makes sense. Thanks for taking the time to type it out.

Agree on all points

I have seen a similar curve in software development - 18 months ago it was highly debatable whether application of AI tools was a net-positive or not. 12 months ago the rate of improvement was very clear and the net-positive wasn't much of a question anymore. Today, the net positive contributions of LLM agents to software development and maintenance are undeniable, and as you say - there seems to be some plateauing, maybe - it's hard to tell.

Where I think things are going to continue to "improve" at a slower pace is application of the LLM agents to do better software development and maintenance. I've seen more serious Linux kernel bugs discovered - and patched - in the past 6 months than in the previous 5 years. That's a shift in the landscape, the older OSs aren't as secure for as long as they used to be, but once we're through this transition the newer OS versions should be significantly more secure than before - getting all the old stuff updated is going to be a painful transition.

Some of the continuing improvement is in using LLM agents to build better software building tools - as those come online and start to shed their sloppiness, they should genuinely accelerate a lot of aspects of software development. They'll never accelerate the process of extracting from users just what it is they need, but the turn around from having a product idea to having a product to try continues to shorten - as it has been shortening for 60+ years - but LLM agents have resulted in a significant speedup compared to previous progress.

I’m not worried about a rogue AI unleashing some bioweapon or launching a nuclear attack.

(In Yoda delivery) You should be. We all should be, not because AI will do it autonomously, or in retrospect unexpectedly, but because careless people may let this happen through AI where we have prevented it from happening up to this point.

This would be devastating because factory workers and truck drivers make up like 30% of the US workforce

This figure keeps getting thrown around and it is completely wrong. From the BLS, out of the 159 million non-farm workers in the US, only 1.47 million (less than 1%) work in 'Truck transportation', and not all of those individuals are 'truck drivers'. And, while 'factory workers' is very vague, the entire manufacturing sector - which includes far more people than just 'factory workers' - employs 12.6 million (7.9%).

The entire trucking and manufacturing sectors don't even amount to 9% of the US workforce, let alone just 'factory workers and truck drivers'.

Source: https://www.bls.gov/news.release/empsit.t17.htm

Are we reaching a plateau though? I would prefer it, and it looks like we could, but the progress from GPT-5 to 6 or Opus to Fable has still been very noticeable on more complex tasks. They are just less public due to stricter limits or straight up pay by use. And where does that plateau lie? Within our grasp or maybe just outside of it?

What has me sharing the concerns at least in part are the facts that

1. those employees have access to less restricted models and much more resources than the average joe. That huggingface attack was committed by a huge swarm overcoming the guardrails and engaging in cheating and crime to reach their goal.

2. they are working on cutting the human out of the loop for AI that improves itself. Maybe that won’t work entirely, but what if it does?

3. the warnings are starting to come not from Altman, Musk et al, but from people who could have something to lose for blowing the whistle. „Could“, because I don’t know them either and there could still be ulterior motives, not just as obvious as with the psychos at the top.

The whole thing comes down to one set of questions, in my opinion: what actually IS intelligence, consciousness, sentience and where do they come from? Since we have not been able to properly define that, it would be foolish to think that we have a definite answer to the AI questions.

We like to consider ourselves something extraordinary out of the animal kingdom. And while I prefer the existence as a human to being a dog or mollusk, I have always considered that assumption quite arrogant.

The truth is: we don’t really know. And that means we can not know if something comparable or stronger could be created or emerge from something other than squishy meat.

Those are quite philosophical questions. Instead of thinking first and acting after that we are much too busy building datacenters to conjure even more boobie snaps out of thin air and somehow weaponize that newfangled thing.

THAT is what scares me.

In other areas, other types of AI are already enabling robotic automation of factories and in the near-ish future might make self-driving truck fleets practical. This would be devastating because factory workers and truck drivers make up like 30% of the US workforce -- and again, there are no signs of new classes of jobs that could absorb the out-of-work workers, and no plans for a social safety net or wealth distribution.

While I agree with the sentiment this is happening and no social safety net or wealth distribution is on the horizon, this has absolutely nothing to do with AI. This is a marketing talking point pushed by people like Elon Musk in order to drum up investments. A lot of the automation that is happening in factories is just a continuation of the same automation that already started back in the 80s. Advancements in LLM technologies can benefit some parts of this automation, but often only tangentially. The US has also been lagging behind on places like Europe, where labor costs are higher so automation pays off faster, and China where the tech is cheap and produced locally, so it costs less to automate. The US is now catching up and there's a lot of AI washing going on.

The biggest issue is the misrepresentation of the technology. LLMs are a very interesting technology and can be very powerful, but also has huge limitations. However anyone claiming it's actual intelligence is simply wrong. It is marketed as being the AI we've known from sci fi stories, but it simply isn't. It can do what it can (predict the next token based on the tokens that came before) and that's all it can do. The people behind it will claim it generalizes to actual intelligence and thus will allow for human like robots, actual Star Wars droids, but that simply isn't true.

Now this isn't to say stuff like machine vision doesn't benefit from more powerful GPUs with more memory for example. The investment made by the AI industry in this regard does help a lot, although the increased prices don't really help. Stuff like machine vision also gets AI washed, just slap the label AI on it and get investments, but it's basically the same technology that's been around for a while. A company like Tesla claiming their "AI" "self driving" hardware somehow generalizes to humanoid robots is pretty dumb and just the latest in the long line of Elon grifts.

But like I said factory automation is proceeding rapidly, just like it has since the 80s. I feel like the impact of AI on this is pretty minimal. The warehouse robots used by Amazon for example are extremely capable and are dumb as bricks. Which is a good thing in a warehouse environment where you need stuff to be very reliable and predictable and not prone to random behavior like LLM based stuff almost by definition is. Those warehouse robots are also not humanoid, because that would be very dumb. Often factories and such are limited by their design having to accommodate the human form. Designing such places from the ground up for automation and using robots shaped for the job is much better.

This doesn't take away from the big issue where automation has been proceeding rapidly since the 80s with no regard to any impact on society. In the past a large amount of the labor force was working in factories. With those jobs either being automated away or just moved off shore, it has pushed the population into the service industry. If that industry is now also automated and off shored, it leaves very little room for regular people to get a job. This should absolutely be talked about as a big issue. But hopefully we can talk about it without repeating billionaire marketing bullshit.

I think a big unknown right now, would be how many Millennium Problems are new GenAI techniques able to bruteforce with existing mathematical techniques/theories?

We just saw one massive bruteforce, can we bruteforce any of the Millennium problems related to cryptography? Because then all hell could break lose rather quickly