588
Google is cannibalizing the web to feed AI (www.theregister.com)
submitted 4 months ago by to c/technology@lemmy.world
138 comments fedilink hide all child comments
you are viewing a single comment's thread
view the rest of the comments
[–] 188 points 4 months ago (49 children)

The death of Stackoverflow is one of these events where the site has been completely killed by AI and yet its contents is completely necessary for AI to know about solving programming problems. Its death will mark the end of AIs ability to learn how to solve programming issues. Its cannibalizing itself in the process, as it destroys its sources it destroys its own ability to learn.

  • permalink
  • fedilink
  • source
  • hideshow 49 child comments
    •
  • [–] 95 points 4 months ago* (46 children)

    It's not just that, it's shitting where it eats. People are using it to fill the internet with disinformation, then it trains itself on it's own disinformation, and breeds even worse disinformation. This is why AI can never be smarter than it was in 2021.

    On top of that, due to the indiscriminate DDOSing of the entire internet by AI bots, websites have been blocking any web crawlers that are not Google, which just contributes to their monopoly.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 46 child comments
    •
  • [+] -7 points 4 months ago* (6 children)

    This is why AI can never be smarter than it was in 2021.

    You're completely wrong. First of all, datasets are getting bigger and objectively better. Secondly, technologies and methods of model training become dramatically more complex. Yes, AI content can create echo for itself, but it's a solvable issue.

    Anyone who disagrees with this fact should do a reality check and learn a little more about the current AI state. In 2021 we didn't even have reasoning in models.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 6 child comments
    •
  • [–] 4 points 4 months ago* (1 child)

    The model architecture will still improve obviously. But the quality of the training data has been irreversibly altered and will only get worse with time.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 3 points 4 months ago* (last edited 4 months ago)

    It would be true if dataset’s composing mechanism was nothing but a blind internet scraping.

    But it’s not so simple. There’s many hundreds of quality and trust filters that were created solely to prevent such things. As I said, it’s a solvable issue, even if most folks here really believe that the smartest AI was buried somewhere in 2021

  • permalink
  • fedilink
  • source
  • parent
  • [–] 1 point 4 months ago* (3 children)

    I see someone's been suckling the OpenAI teet.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 3 child comments
    •
  • [–] 2 points 4 months ago* (last edited 4 months ago) (2 children)

    What it have to do with OpenAI? It’s not what I’m talking about.

    Nonetheless, it’s kind of fun to see so much downvotes on my comment. Some people are so blinded by their hate (well deserved, by the way) that they will deny something that obvious.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [+] -19 points 4 months ago (6 children)

    Model collapse isn't a thing anymore. https://arxiv.org/html/2510.16657v1

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 6 child comments
    •
  • [–] 29 points 4 months ago (2 children)

    Our key finding is that by injecting information through an external synthetic data verifier, whether a human or a better model, synthetic retraining will not cause model collapse.

    Yeah if you have a source of truth then your model is basically getting trained on that.

    It’s like already having the answer

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 21 points 4 months ago*

    Our key finding is that by injecting information through an external synthetic data verifier, whether a human or a better model, synthetic retraining will not cause model collapse.

    Lol, so to make a great model, they just need to have an even better one available first or a human who can verify every single thing it ingests.

    Hmm, call me skeptical on this claim.

  • permalink
  • fedilink
  • source
  • parent
  • [–] 10 points 4 months ago

    This assumes everything is valid on the external. If one slop cluster feeds off another - a slopveyor? - then there is nothing external for the validation hall-monitor to compare against. They're trusting another model's output as if it were gospel.

  • permalink
  • fedilink
  • source
  • parent
  • [+] -22 points 4 months ago (31 children)

    I'm pretty sure AI is objectively smarter today than it was 5 years ago.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 31 child comments
    •
  • [–] 13 points 4 months ago (4 children)

    How can something with no intelligence be smarter?

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [+] -7 points 4 months ago (3 children)

    It is evident that it has intelligence, it outputs intelligent responses usually adequate to its input, even if it’s badly phrased. What it doesn’t have is sentience, conscience, and a learning loop.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 3 child comments
    •
  • [–] 1 point 4 months ago (2 children)

    Evident to whom exactly?

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 0 points 4 months ago (1 child)

    To anyone who interacts with it? Would you deny that a program automate mental labor in the same way that a sawmill automates manual labor? Isn’t that some degree of intelligence?

    Now, we have very imperfect LLMs who nevertheless can be instructed not with program code, but actual natural language, and they react accordingly. Isn’t that also intelligence? Computers that understood natural language was the realm of science fiction just five years ago.

    I get it that people hate LLMs, both because how idiots use them, and how corporations push them everywhere; but not recognizing the intelligence in those programs is naive at best.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] -2 points 4 months ago* (last edited 3 months ago) (19 children)

    This is true, depending on what you mean by smarter. They are undeniably more capable. However, the trendy, cool thing is to hate on AI, rejecting all else. Sure, capitalism sucks, and the powerful rich people and companies who control AI suck. AI itself, though, can very easily result in massive benefits for humanity as a whole.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 19 child comments
    •
  • [–] 3 points 4 months ago (16 children)

    "AI" is garbage generator, that consumes our resources to output mediocre slop. Case in point is your PFP.

    It's neither "trendy" nor "cool", just common opinion, as more and more people get to try it and see how useless it actually is, for anything beyond average (often empirically wrong) output.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 16 child comments
    •
  • [–] -1 points 4 months ago (15 children)

    Recent LLMs are very capable at doing almost all simple computing tasks. At some point in the future, we will have AI that is more generally useful than current LLMs. If and when we have truly general purpose AI, it could benefit humanity in countless ways. It could also continue to cause harm. We can’t know for sure.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 15 child comments
    •
  • [–] 2 points 4 months ago (11 children)

    Full self driving next year.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 11 child comments
    •
  • [–] 0 points 4 months ago (10 children)

    We already have it

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 10 child comments
    •
  • [–] 1 point 4 months ago* (9 children)

    Sure you do. In your colony on Mars.
    How fucking gullible can you get, jfc.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 9 child comments
    •
  • [–] -1 points 4 months ago* (8 children)

    I’ve been in vehicles with self-driving, and they perform better than the average driver in every situation I’ve encountered.

    You seem like a really angry person. I’m sorry your experiences in life are so miserable.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 8 child comments
    •
  • [–] 1 point 4 months ago (7 children)

    Wow, even a personal anecdote! That's, like, irrefutable! (since you didn't really understand what I'm saying, I'll clarify - this was a Musk/Tesla reference)

    Listen, next year I'm curing cancer and solving world hunger, thus making ""AI"" obsolete. Care to invest?

    Until then, it's best if we evaluate technology at its current standing. Which is garbage generator that requires way too much resources.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 7 child comments
    •
  • [–] 0 points 4 months ago* (last edited 4 months ago) (6 children)

    A personal anecdote is all that’s needed to refute a blanket statement like the implication that self-driving does not exist yet.

    Again, it seems you are just a really upset and negative person. I feel bad for you. It would be very easy to have a pleasant discussion or debate on the topic.

    If we only ever evaluated technology on its present state of being, we wouldn’t have planes for instance. That’s not a really sound methodology. “It doesn’t work now. Therefore, it should be abandoned.” Like, what?

    Though, perhaps what you mean is “we shouldn’t be investing billions in unproven technology.” That, I agree with. I think most would.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 6 child comments
    •
  • [–] 1 point 4 months ago (5 children)

    It seems that youre just a really dumb and uneducated person. I feel bad for you.

    Now that we've got ad-hominems out of the way, let's get back on the topic.

    Damn, I really thought I explained the self-driving thing, but it just bounced off you, didn't it. Reaping the effects of AI so soon, huh.

    If we only ever evaluated technology on its present state of being, we wouldn’t have planes for instance. That’s not a really sound methodology. “It doesn’t work now. Therefore, it should be abandoned.” Like, what?

    If you think that's how design and engineering works, you're either 12, or really into LLMs.

    Look, I'll put it really simply for you. I know that if I throw a ball forwards, it will land in front of me. I call predict it, and I can calculate it. It's based on established laws of physics.

    If people, who want to sell me a pig, tell me it's going to fly next year after I buy it, based on the fact that it learned to walk, that is not a prediction based "on sound methodology".

    Enjoy your slop, it's not going to get any better, and it's going to get ALOT more expensive very soon. That's a prediction, after evaluating current technology.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 5 child comments
    •
  • [–] 1 point 4 months ago* (4 children)

    I am far from dumb. I got 60/60 problems correct on a proctored Raven’s Progressive Matrices as a teenager. That places my pattern recognition much higher than most of the world’s population. I taught myself C++ when I was six years old in 1997.

    You’ve created a strawman by recasting my argument to a point that I am not making. I did not say “trust vendor promises” or “ignore current failures.” I said present-state-only evaluation is a bad way to judge emerging technology.

    Engineering forecasting is not limited to closed-form physics. It also uses empirical progress, deployment data, cost curves, failure modes, and observed capability gains. Your pig analogy only works if AI had no demonstrated utility and no measurable improvement. That is plainly false.

    Criticize cost, energy use, incentives, etc. Those are real arguments. But “some promises are hype, therefore the whole field is slop and cannot improve” is not analysis. It is just cynicism with insults attached.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [–] 0 points 4 months ago (3 children)

    Wow, I'm very impressed. Reads like billionaire's wiki page.

    Analogy absolutely works. The pig could not walk when it was born, now it can. Which means that next year it'll fly.

    Exactly the same type predictions are advertised for LLMs. It can't count to 10, but a text predictor will be able to find cure for cancer.

    If you're as smart as you claim you are, you'd go read a few papers on the limitations of the technology as a whole. "It will get better" / "It's in its infancy" is just pure bullshit. There will be optimizations and gains here and there. But the core technology is not capable of producing what is being promised.

    Pig will not fly, I'm affraid, even though it might be able to make do a hop next year.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 3 child comments
    •
  • [–] 1 point 4 months ago* (last edited 4 months ago) (2 children)

    That’s some pretty absurd logic. It is clear there is no point in continuing this discussion, as you are too easily trapped by logical fallacies and are too quick to jump to insults. Hopefully you can take some time to try to grow as a person.

    The field of Machine Learning and Artificial Intelligence does not depend on the success of Large Language Models, and LLMs quite possibly have reached their limits. I do not see that as relevant.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 0 points 4 months ago (1 child)

    Nice goalpost move.

    "In the future, "AI" will be a completely different technology".

    Lol, why do I even bother discussing things with techbros... Spare the "insults" part, you're the one who started that.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 1 point 4 months ago*

    No, it will be the same field of technology. LLMs are just a facet of AI, currently. I’m done having this discussion with you, as you do not have the capability to grasp rational thought or have a rational discussion.

    And, no. You started the insults. I responded with a couple.

  • permalink
  • fedilink
  • source
  • parent
  • [–] 1 point 4 months ago (2 children)

    I know, for sure.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 1 point 4 months ago* (1 child)

    You truly don’t, though. You believe it. That doesn’t make it true. We do not have enough data to know what the future holds regarding AI.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 1 point 4 months ago*

    At some point in the future, we will have AI that is more generally useful than current LLMs. If and when we have truly general purpose AI, it could benefit humanity in countless ways

    -- You. Just now.

  • permalink
  • fedilink
  • source
  • parent