Ok. I haven't tried it so I'll take your word for it. I'm just offering an easier alternative since the topic was "getting started"
hyperhypervisor
joined 7 months ago
Iirc it can run anything llama.cpp can because it just uses that under the hood.
There's also llamafile, super simple: download and run it.
Markdown is not respected in post preview (can't attach an image apparently, but many posts on news communities will have a quote)
No, they just mention that only Apple silicon is supported if you're using MacOS