▲ 1801 ▼ might be a form of Jevons Paradox (thelemmy.club) submitted 6 months ago by not_IO@lemmy.blahaj.zone to c/programmer_humor@programming.dev 290 comments fedilink hide all child comments https://raru.re/@ocean/116214437024389203
[–] Samskara@sh.itjust.works 2 points 6 months ago (2 children) Have you ever tried running a decent sized LLM locally? permalink fedilink source parent hideshow 2 child comments replies: [–] Vlyn@lemmy.zip 2 points 6 months ago (1 child) Decent sized for what? Creative writing and roleplay? Plenty, but I try to fit it into my 16 GB VRAM as otherwise it's too slow for my liking. Coding/complex tasks? No, that would need 128GB and upwards and it would still be awfully slow. Except you use a Mac with unified memory. For image and video generation you'd want to fit it into GPU VRAM again, system RAM would be way too slow. permalink fedilink source parent hideshow 1 child comment replies: [–] Samskara@sh.itjust.works 1 point 6 months ago I use a Mac with unified memory, so that distinction slipped my mind. permalink fedilink source parent
[–] Vlyn@lemmy.zip 2 points 6 months ago (1 child) Decent sized for what? Creative writing and roleplay? Plenty, but I try to fit it into my 16 GB VRAM as otherwise it's too slow for my liking. Coding/complex tasks? No, that would need 128GB and upwards and it would still be awfully slow. Except you use a Mac with unified memory. For image and video generation you'd want to fit it into GPU VRAM again, system RAM would be way too slow. permalink fedilink source parent hideshow 1 child comment replies: [–] Samskara@sh.itjust.works 1 point 6 months ago I use a Mac with unified memory, so that distinction slipped my mind. permalink fedilink source parent
[–] Samskara@sh.itjust.works 1 point 6 months ago I use a Mac with unified memory, so that distinction slipped my mind. permalink fedilink source parent
