Vlyn comments on might be a form of Jevons Paradox

[–] Vlyn@lemmy.zip 17 points 6 months ago (3 children)

With 32 and 64 GB systems I've never run out of RAM, so the RAM isn't the issue at all.

Optimization just sucks.

parent

[–] Samskara@sh.itjust.works 2 points 6 months ago (2 children)

Have you ever tried running a decent sized LLM locally?

parent

[–] Vlyn@lemmy.zip 2 points 6 months ago (1 child)

Decent sized for what?

Creative writing and roleplay? Plenty, but I try to fit it into my 16 GB VRAM as otherwise it's too slow for my liking.

Coding/complex tasks? No, that would need 128GB and upwards and it would still be awfully slow. Except you use a Mac with unified memory.

For image and video generation you'd want to fit it into GPU VRAM again, system RAM would be way too slow.

parent

[–] Samskara@sh.itjust.works 1 point 6 months ago

I use a Mac with unified memory, so that distinction slipped my mind.

parent