138
The price of AI is crashing faster than the rate of Moore's Law — intelligence costs are in freefall, outpacing comparative technologies like compute, DNA sequencing, and lithium batteries (epoch.ai)
submitted 2 days ago by to c/technology@lemmy.world
65 comments fedilink hide all child comments
 

cross-posted from: https://piefed.world/c/tech/p/1434965/the-price-of-ai-is-crashing-faster-than-the-rate-of-moore-s-law-report-suggests-intellig

you are viewing a single comment's thread
view the rest of the comments
[–] 3 points 1 day ago* (4 children)

All that R&D, and they need to squeeze efficiency out more at this stage. Faster and more specialized is better than big chingus do everything, and means we can hopefully stop spending all that money on making more datacenters.

Personally, I don't use the big chingus models at all... I much prefer local gen if I use it for things. Much better privacy that way, and I'm not throwing money at these jokers.

  • permalink
  • fedilink
  • source
  • hideshow 4 child comments
    •
  • [–] 1 point 1 day ago (3 children)

    Are there some local models that one can use that are not giving our data away and on par with chat GBT?

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 3 child comments
    •
  • [–] 2 points 1 day ago* (2 children)

    Ollama, Open WebUI, and Qwen, or one of the edits of it. The latest Qwen 26B is in a very good delta of smart enough to know how to help with stuff (I use it as a second opinion/initial proofread code checker) and small enough to run on a macbook or a gaming pc, and Open WebUI is the self host ChatGPT-like interface that works with Ollama, the turn key ai model runner backend. Couple that with your own searxng instance self hosted to give it the ability to search the web for answers if you want to give it that. Run it all in a virtual machine, of course.

    For media content, wan2gp (wan2 for the gpu poor) has an all in one stop shop for self hosted audio, image, and video. Just beware people don't like generated stuff in prod, but its useful for quick concepts, or to do certain types of edits of existing stuff.

    There's also tts and speech recognition via the whisper models, which sounds a lot better than the speak and spell voices from the past (unless you enjoy the degeneracy of spamming JOHN MADDEN JOHN MADDEN in the Steven Hawking voice) and can transcribe words a lot faster (realtime) and more accurately than in the past. The best ones run via Vulkan and thus work on any card that supports Vulkan.

    If you run Nextcloud, you can also connect Ollama to it to get the Nextcloud AI working.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 1 point 11 hours ago (1 child)

    Great input, thank you I appreciate it. Next off for me is learning how to do a virtual machine! I do have ollama currently on a computer connected to a speech to text app. And I find that very useful to have

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 2 points 9 hours ago*

    Virt-manager for a gui, qemu-kvm and libvirtd for the hypervisor. You will need to pass thru a graphics card, which makes it unavailable for the host machine entirely. I use debian stable for the vm, no gui, just ssh to manage it.

  • permalink
  • fedilink
  • source
  • parent