you are viewing a single comment's thread
view the rest of the comments
[–] 2 points 2 years ago (1 child)

I don't know of an LLM that works decently on personal hardware

Ollama with ollama-webui. Models like solar-10.7b and mistral-7b work nicely on local hardware. Solar 10.7b should work well on a card with 8GB of vram.

  • source
  • parent
  • hideshow 2 child comments