Sure, we could do that. But we'd need even more computer hardware that we don't have, so it would be more expensive, and energy use would be less efficient so there'd be even more environmental impact. Just spread around to every house rather than in one centralized easily-protested spot.

Sam Altman was incredibly tone deaf and it was a stupid thing for him to say out loud when he said RAM was better off in data centers, but in some significant and important ways he was right. Computer equipment is much more efficiently utilized when it's in a data center.

I say this as someone who's super into running local models myself. I do it because I believe the autonomy and privacy are worth trading off the extra cost in hardware and electricity. If I just wanted to use AI to get something done and didn't care about that I'd be paying for API access instead.