I'd definitely like someone to champion an open-weights model under 32 GB. But part of that desire is - I'm not sure they're there, yet. Local models are only a few months behind, but hoo boy, the last few months have been big.

Coding models could be tiny, but distilled versions keep preserving pop-culture knowledge and world history, when reasoning through C++ requires essentially none of that. It needs the tokenization to recognize why a comment begins "Jesus Christ." It's fine for "treat the hash like Lot's wife" to go completely over its head. Keep the weights that know a regex for e-mails, not the capitol of Florida. If it speaks a dead language I want it to be Algol.

I am wondering how many undetected bugs will sneak into the kernel.

Less with LLMs than without. https://projectdiscovery.io/blog/the-vulnerability-curve-bent-with-the-ai-curve https://www.darpa.mil/news/2025/aixcc-results

You wouldn't want him using a local model unless he was running some fancy rig that could run a high end frontier model. He might not have such a thing laying around. Using the best out there for something like this is how you help avoid bugs.