The article talks a lot trash about AMD and ROCm but vulkan works fine too. In fact from a datacenter GPU standpoint there is an AMD option called the V620 available on US EBay that I was able to haggle to $350, with 32GB VRAM, 512GB/s bandwidth, and runs the same Qwen-3.6-27b at about 20t/s. I would argue that's even more cost effective.
It requires a few of the same fan shenanigans this guy did but there is no need to pull specific past software versions to make it usable in Linux

Here I was expecting a graphics demo to blow our collective minds but instead I got a story about a local LLM for cheap. It is . I should have known better.

Were I the author / tech cobbler here, I'd be concerned that too much time with an LLM, local or otherwise, might erode or dull my apparently fairly sharp reasoning and tech skills. (Clarification: Not my sharpness, theirs. I'm a potato.)

Other thoughts: For a minute I thought this whole thing was a tribute to, or a troll in the manner of, that one Redditor that always spun their stories around to being about their dad beating them with jumper cables.

Also, my old PC developed an issue like the warm reboot problem, except with the network interface. I couldn't just restart, I had to power off and back on. I never did bother to find out whether it was early signs of hardware failure or whether it was an old hardware / newer kernel mismatch.

Seems like an awful lot of trouble to save $100 not buying a 5060 Ti that also has 16GB.

eBay has some rad Chinese mezzanine boards for these guys too. Nvlink works and everything lol

Sure it's got a lot of VRAM, but the 4080 has five times the compute power.

Why tho?

The main thing that itches me with the V100 is the fact that given that pascal is about to be EOL, a 2017 card is probably soon next

82db? Wild. I guess they don't really care that much about noise in data centers though.

Yeah but the 0 point doing this unless you want to run AI models for some reason. These GPUs can't do video game graphics so this isn't a solution to the GPU shortage.

This is a bit like me writing an article about NASCAR, now I can turn left whenever I want. But I haven't magically acquired a functional vehicle for a fraction of its value. I've purchased a second hand specialist product that is usually useless outside of that environment.

Not an AI guy, but I do like using niche hardware wrong to get results cheap. Can anyone tell me what this would be like for gaming or general computing? My 1660 super was a budget pick when I got it back in '18.