Yeah I'm just beginning my local AI journey on a 5080, tried Qwen3.6 27b Q4 and was getting like 1tps because of the vram overflow. Ran it over night at it was still chewing on generating a prompt for a sub agent when I got up in the middle of the night until it simply ended in some kind of "fetch failure" lol. I think I gave it something too large to tackle, but either way 1tps is kinda garbage.
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
replies: