I felt sorry for it and stopped it, 14min and the poor thing was just going around in circles.

I'm just looking at running some local llm on my m2 16GB mac and this was with the default settings for the qwen3.5 9b in LM Studio. Any pointers to get the best out of a M2?

you are viewing a single comment's thread
view the rest of the comments
[–] 4 points 4 hours ago (3 children)

Try Ternary Bonsai. It's a heavily quantized version of Qwen 3.8 27b. It won't do wonders, but it shoud be better: https://huggingface.co/prism-ml/Ternary-Bonsai-2-27B-gguf-dev The page says you need a fork of llama.cpp, but the support was included in the latest release (0.7). You can read more about the mode on the main model page: https://huggingface.co/prism-ml/Ternary-Bonsai-27B-gguf

  • source
  • hideshow 3 child comments