654
Motherboard sales are now collapsing amid unprecedented shortages fueled by AI (www.tomshardware.com)
submitted 3 months ago by to c/technology@lemmy.world
146 comments fedilink hide all child comments
 

cross-posted from: https://lemmy.bestiver.se/post/1091783

Comments

you are viewing a single comment's thread
view the rest of the comments
[–] 12 points 3 months ago* (1 child)

I've looked into self-hosted AI and decided it's not worth the cost - both in terms of hardware and energy - when compared to the relative value to be gotten out of it. YMMV.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 1 point 3 months ago (1 child)

    Same, pretty much. It is possible though, which makes LLMs a more democratic technology than, say, nuclear reactors.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 3 points 3 months ago* (1 child)

    The models you can run on consumer hardware are still nowhere near the stuff that runs in corporate data centers. To stick with your metaphor, its like running a little steam engine at home while the big guys get to operate nuclear reactors...

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] -2 points 3 months ago

    You can get pretty far with a stack of 5090s and llama.cpp with split mode graph (or so I've heard, I've never tried), or AMD's unified memory CPU thing.

    It's not as good as data centre grade stuff, but it's not nothing either.

  • permalink
  • fedilink
  • source
  • parent