339
Nobody Asked for AI (jacobin.com)
submitted 1 month ago by to c/technology@lemmy.world
98 comments fedilink hide all child comments
you are viewing a single comment's thread
view the rest of the comments
[–] 17 points 1 month ago (15 children)

Self-hosted LLMs are neat and private, if you (already) have the hardware/budget and some cheap (solar PV) surplus juice to drive it.

  • permalink
  • fedilink
  • source
  • hideshow 15 child comments
    •
  • [–] 2 points 1 month ago (6 children)

    Do you know anyone doing all of that? Running a local model on off-grid power?

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 6 child comments
    •
  • [–] 4 points 1 month ago* (last edited 1 month ago) (2 children)

    What would you like to know about it?

    I glossed over it the other day here -

    https://aussie.zone/comment/24336560

    From go to woah (including the compute), total project cost is about $1400 USD / ~$2K AUD. I realise I have the privilege of claiming that's not a lot, but for a hobby spend that makes me happy / does useful stuff (see TV example above), it's not a lot.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 1 point 1 month ago (1 child)

    It just sounded like the kind of thing that makes good sense on paper but I’m wondering if it actually works out well. Sounds like you are in the process of getting ready to try it.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 2 points 1 month ago* (last edited 1 month ago) (2 children)

    Someone mentioned that, but it doesn't have to be off-grid. My 2 kWp system is grid-tied and if I would start running a local LLM it would be an incentive to add another couple.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 1 point 1 month ago (1 child)

    It may sound odd but I’ve heard people bring up the above point a few times. “AI isn’t bad - if you run it on a sustainable off-grid system it’s perfectly fine.”

    But I have NOT encountered people actually doing this even once, let alone a few times. That’s why I was asking.

    Anything that wastes energy can be recast as harmless if you put it on an imaginary sustainable setup but that just seems like fantasizing.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 1 point 1 month ago

    Renewable infra isn't sustainable, since it can't be built with renewable energy alone. As such it's just a fossil multiplier. I make about 2/3rd of my own net power, since my electricity meter is old enough to run backwards. I can't install more orelse my meter would read negative. I intend to run a Qwen model that fits my 8 GB HBM Radeon card as an experiment during the day, starting end of this month.

    If this works I consider running something hotter, and double my PV capacity. Adding AC or an EV would be another option. None of this is sustainable, but not exactly a fantasy.

  • permalink
  • fedilink
  • source
  • parent
  • [–] 1 point 1 month ago (7 children)

    oh so yours is trained on ethical sources? you compensated the creators for their work and did so with their consent to machine training?

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 7 child comments
    •
  • [–] 1 point 1 month ago* (last edited 1 month ago) (6 children)

    What the fuck are ethical sources? Where is compensation and consent when I unapologetically use pirate archives of books and papers? You people crack me up.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 6 child comments
    •
  • [–] 1 point 1 month ago (5 children)

    the fact that you have no idea what I'm talking about either means you're full of shit, or cartoonishly stupid and evil.

    Probably both

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 5 child comments
    •
  • [–] 1 point 1 month ago (4 children)

    The fact that you immediately descend into ad hominems indicates that you're a just another troll that should be posting from hexbear.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [–] 1 point 1 month ago (3 children)

    you're simply used to llm sycophancy, not talking to actual people.

    you really not understanding training on sources compensated for their content is hilariously stupid

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 3 child comments
    •
  • [–] 1 point 1 month ago (2 children)

    You're not understanding what standing on the shoulders of giants means. Locally hosted open weights models are free/libre and puts you in control of your data. Since we're talking about hilariously stupid.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •