213
AI’s Memorization Crisis | Large language models don’t “learn”—they copy. And that could change everything for the tech industry. (www.theatlantic.com)
submitted 8 months ago by to c/technology@lemmy.world
45 comments fedilink hide all child comments
you are viewing a single comment's thread
view the rest of the comments
[–] 9 points 8 months ago (2 children)

Yep this has happened to me too

I used to always get the results I was looking for now it’s just pure garbage but Gemini will have all the expected results as sources

Obviously deliberate to force us to use Gemini and make free searches useless. Maybe it hasn’t been rolled out to everyone yet but it’s certainly got us

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 3 points 8 months ago (1 child)

    I'm honestly not even sure it's deliberate.

    If you give a probability guessing machine like LLMs the ability to review content, it's probably just gonna be more likely to rank things as you expect for your search specifically than an algorithm made to extremely quickly pull the most relevant links... based on only some of the page as keywords, with no understanding of how the context of your search relates to each page.

    The downside is, of course, that LLMs use way more energy than regular search algorithms, take longer to provide all their citations, etc.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 2 points 8 months ago

    A ton of factors have increased energy costs on the web over the years. It's insignificant per person, but bandwidth is exponentially higher because all websites have miles of crud formatting code nowadays. Memory usage is out of control. Transmission and storage of all the metadata your web browser provides in realtime as you move your mouse around a page is infinitely higher than what we had in the early days of the web.

    The energy cost of ML will reduce as chips progress, but I think the financial reality will come crashing down on the AI industry sooner rather than later and basically keep it out of reach for most people anyway due to cost. I don't see much of ROI for AI. Right now it's treated as a capital investment which helps inflate company worth while it lasts, but after a few years the investment is worthless and a giant money sink from energy costs if used.

  • permalink
  • fedilink
  • source
  • parent