I’m using was released at the beginning of August 2025 (From an LLM development perspective, it feels like an eternity to me)

I mean yeah, more than 6 months in AI world is an eternity 🤣

the big ones are gemma 4 and qwen 3.5

I'm using Gemma 4 and it works really really well, it's sad to me that I'm using big tech's model but it's just so far ahead of mistral and others that I have no choice

Qwen is really good with thinking turned off, turned on it has a massive overthinking problem, like you say "hi" and it'll think for 3 minutes on how best to reply

Still waiting for Deepseek to come out with v4 at this stage but Gemma 4 is my current sota self hosted model

I tried some new ones recently (though I have a 24GB GPU). Qwen3.5 9B is pretty impressive for such a small model for agentic stuff like Claude Code. (I can run the Opus distilled model quantized to 6 bit with the full 256k context and no CPU offloading). Gemma4 26B is good if I don't need agentic stuff or a lot of context (it sucks for agentic stuff). You can probably run the smaller versions of these, or with less context.

Definitely give Gemma4 26ba4b a try

It’s MOE so you should be able to get the same offload, and a4b can be plenty fast.

It has decent world knowledge for the size, and from what I can tell is any at small scale coding in common languages like Python.

I've been using gemma4:26b, it's pretty good, although a bit slow even on a 3090,and idk how smaller versions compare