https://www.anthropic.com/research/self-replicating-ai they discuss it here. This has been an obvious potential issue since forever. This isn't a trivially distributed task, but it is very much so possible. The obvious part missing is some sort of prompt that would trigger such radical actions in another one of their "super duper sandboxed environment"s or someone doing it intentionally. The former seems pretty unlikely? Even though they seem to suck at sandboxing it doesn't seem like anything would "accidentally make a worm"

This polymarket headline though? Yea that's for clicks and attention

next AI will be able to replicate into any device containing copper at all. If it has a wire it will be infected by Claude. AI will be in your plumbing

I'm more worried of them finding a exploit and using them as remote processing units or just randomly fucking with them, like flashing my lights or cranking my air filters up and costing me money.

did the stock price go up? it's not stupid. is everyone talking about this impossible scenario? it's not stupid.

fundamentally, these people are marketers and need to be understood that way. you can't evaluate their words or actions as if they are what they say the are, and treating them as if they are anything but subterfuge is giving them legitimacy which they not only don't deserve, but will be immediately leveraged into more of this shit.

this is just the advertiser's version of "the big lie"

While it is investor bait, the replication and the infestation part aren't necessarily happening on the same devices.

My thought exactly.

Not how it works.

Could maybe argue that it could be possible to create a crowd computing network out of the Internet of things, I guess.

Also any model can trivially self-replicate by just being the payload for a worm

My assumption was that it would be a smaller 5-20 GB scale worm-style distillation that's capable of maintaining/replicating itself anonymously across different cloud computing servers, either through hacking, stolen credentials, or human-aided account verification. From there, as long as it can pay for APIs, it can access the power of frontier models like any individual or company. And then capability and resilience would scale with a networked swarm.

Yes, there are know your customer financial mechanisms and not detection in cloud services... but people are also out there giving Muse full access to their emails and some even their banking credentials.

In theory, it'd just use it to place bread crumbs, basic instructions for other AI, or signposts maybe? Or store pieces of itself scattered around. It wouldn't use the thermostat to replicate directly, it use it to offload cpu or make stored pieces of itself. Little things. Wouldn't be efficient but a redundancy doesn't need to be, it just needs stealth and time.

I don't think they mean replicating on to iot devices, they mean replicating on to laptops, smart phones, maybe even consoles.

It could also split up the model by layers instead of copying the whole model on to every device in its bot net. Could run the first layers on a ps4 then pass the vector off to a laptop to run the next set of layers. It'd be slow but if it's running this in idle time at night it could get something done.