Do you host your own ML / AI / LLM? What do you use, and what do you use it for?

you are viewing a single comment's thread
view the rest of the comments
[–] 2 points 2 months ago

Yeah, a higher quant would be nice, I actually try not to go below Q5, but you can domino’s so much with 16GB of VRAM and the ddr4 system RAM.

But I must say I‘m pretty impressed by Qwen3.6-35b, not only from its capabilities but also from hardware requirements. MoE for the win I guess.

RWKV sounds interesting, have to look into it, thanks!

  • source
  • parent