[-] plasma8726@lemmy.today 9 points 11 hours ago

This is the only way to date.

[-] plasma8726@lemmy.today 1 points 1 month ago

Thanks! I'll look into this. I'm a bit limited at 12GB of VRAM right now.

[-] plasma8726@lemmy.today 10 points 1 month ago

Thanks for this link. Because of this article, I had claude stand up a llama.cpp container next to my already running ollama container. It ran side by side tests with the same model and parameters, and the results blew ollama out of the water. I'm in the process of moving hermes and openwebgui over to the llama.cpp instance to see how it goes day to day.

[-] plasma8726@lemmy.today 2 points 1 year ago

plasma8726

0 post score
0 comment score
joined 1 year ago