Basically the title.

For me, a dense model around 20b, with an additional engram table of 10-20b, or a sparse MoE around 80b would be amazing.

you are viewing a single comment's thread
view the rest of the comments
[–] 12 points 6 days ago (2 children)

gemma 26b-a4b was fantastic, anything in that area is ideal

running my tiny rx6600 8gb vram

  • source
  • hideshow 2 child comments
  • [–] 1 point 6 days ago

    I have an M4 Pro 24GB and I still use Gemma 26B-A4B as my primary model. Beats every other model I’ve ever tried locally on most stuff except coding and tool use, so a Qwen 4 at this size would be super interesting

  • source
  • parent