A society where AI “drives innovation” will become technologically stagnant, because it literally can’t innovate.

This is exactly what Frank Herbert warned us about.

AI may impress the masses, but it reminds me of a cammercial we had on Dutch tv back in 2005. It showed a person driving a car, the navigtion system said "turn right" and the person immediately turned right into the bushes [video]. We laughed at this back than, but this is literally how AI behaves now. It might be getting more suffisticated, but it still operates like this.

yep. and the "ai creates a better ai" is a fallacy because the current ais are limited by the same things the ais they build are. everything the ais are trained on is just snapshots of human knowledge. a microscopic window of the limited reports on human experience which are themselves microscopic reports of a human's lifetime.

When you say "the big argument".. you are mainly talking about the CEOs. They are just auxiliary marketing personnel these days.

AI is changing the world, and will continue. But the ways it does that are drastically different than those CEOs say. I don't need it to think or innovate. I need it to follow the instructions left for it in the repo and just do the things I ask it to. It's getting decent at that. And it really does save me time. Soo much time in tech work is wasted having to go ask 6 people how a thing is done. And everyone comllains that noone write documentation. So now here comes AI. It can write the documentation and follow it. That is a big change. Myself... I used it to fix two bugs in some project that my work was dependent on. I wouldn't have even tried that without AI because I don't know the language, I don't know the many ways the project is used, none of that. I would have been stuck waiting for the team that owns it to fix it. Which surely would have meant escalating to management and all that BS. But AI helped me make the changes and lowered the burden for the owning team so they could actually release the fix. I got unblocked with 10x less effort. That's a big deal. Just don't ask it to think. There is plenty of dumb work to go around.

AI can innovate, but it needs be more advanced than an LLM, which just rearranges existing knowledge. You need some kind of evolutionary loop in its programming. How it continues after that depends on the particular flavour of dystopian sci-fi.

But that assumption is the same as thinking a fallen object will keep gaining speed until it passes the speed of light. Or that Moore's law would really keep going forever despite a very real limit in physics for how small a chip can be.

I gotta say, I've been hearing predictions that Moore"s law is coming to an end every year for the past twenty years.

Look, strictly speaking if you take Moore"s law word for word, sure there is a limit to how small a silicon transistor can be. However, realistically, I see no reason to expect computers won't continue to meaningfully increase in capability every year forever.

When a technology reaches physical limits, we figure out how to bend the rules and get a little bit more out. And when that stops working, we start exploring different architectures, different materials. I think it's just silly to predict that technology won't improve, when has that ever happened?

because it literally can't innovate.

I was with you 100% until this. "Innovation" is super easy. It will always be easy to extrapolate new things from existing things. People just over-romanticize "innovation" as some kind of magic.

I think chances are good that human produced newness will be on average more useful, complete and safe.

Ima be honest, if i was given the task to make a bot to beat another bot tried my best and couldn't. I would just go fork the best bot thats out there and use that as the base and not reinvent the wheel. Learn from its design then improve on it as i study it.

If the goal is ONLY to win a game of starcraft. Fuck it just download the bot. Frankly the fact the bots decided that wasting resources was pointless and to just do the simplier thing is sorta what we want them to do. Thats less eletrical useage, less water wasted, and gets the job done.

A lot of these "the bots cheated or broke the rules" end up always be just because either the rules are not defined, poorly defined or poorly thought out that no reasonable person would follow them either.

And thats the thing these models are litterally the text book definition of wisdom of the masses/mass consensus. They do exactly what the avg run of the mill person with the knowledge at hand would do. That ends up being rather all over the place since people are all over the place. But one of the most common things is that people generally. Hate fucking pointless work and will try to streamline, shortcut or simplify any work given to them to get an acceptable outcome.

Its rare you get someone that will stubbornly work on a problem forever that they can't over come with out changing their apporch rules be damned.