you are viewing a single comment's thread
view the rest of the comments
[–] 5 points 8 hours ago

I switched to LM Studio as my generic "run whatever GGUF I've come across today" platform and I quite like it. It doesn't have the weird setup and configuration thing Ollama does, it's got a nice built-in chat system like Kobold does, and it does dynamic model loading on demand so my various random scripts can call into it for whatever they need without me having to do any manual work.

Running stuff with straight llama would probably be more computationally efficient, but I value easy interfaces too.

  • source
  • parent