Linus Torvalds says AI-powered bug hunters have made Linux security mailing list ‘almost entirely unmanageable’ : linux

[-] lordbritishbusiness@lemmy.world 63 points 3 months ago

I've found that the LLMs tend to over classify and nitpick a fair bit, often missing broader context that accounts for the flaw being tolerated or undiscovered.

They're not wrong, but have no context for triage and so give far too many results. It forces you to consider an LLM subscription yourself just to keep up with the other LLM users which is starting to feel like some form of zero sum red queen's race.

The tsunami of reports won't be receding for a while yet, and we can only hope the teams on the receiving end don't drown in it.

[+] lvxferre@mander.xyz 36 points 3 months ago* (last edited 1 month ago)

[deleted]

[-] fuckwit_mcbumcrumble@lemmy.dbzer0.com -2 points 3 months ago

Is it really a waste of time if the computer can find the bugs, and the dev can focus on fixing them instead of pulling double duty?

[-] luciferofastora@feddit.org 2 points 2 months ago

The human will still need to check and understand the bug.

Particularly if a given bug is reported dozens of times in slight variations, someone will have to check for each of these reports whether it's a bug already reported earlier. If they're all checked by the same person, that person may quickly recognise "Okay, yeah, that's the same thing I logged earlier" but if it's multiple people, there's just so much extra overhead associated with keeping track of what's new and what's "30th reported of same bug".

The AI generating the report probably doesn't check if the issue hss been found before. If the people subvmitting it also don't, you end up with a load jof chaff by lazy people thinking they're helpful when really they're obstructing the efforts.

[-] Lon3star@lemmy.world 36 points 3 months ago

Welcome to AI

[-] Jankatarch@lemmy.world 22 points 3 months ago* (last edited 3 months ago)

It must suck because all chatbot output I seen speaks like a corporate email, just can't get to the point.

[-] spicehoarder@lemmy.zip 5 points 3 months ago

Seriously, I can't stand talking to Chat GPT

[-] Swedneck@discuss.tchncs.de 2 points 2 months ago

one of the big reasons i don't understand how people can use anything other than basic search assist stuff, that's at least directly rephrasing stuff like wikipedia and thus unlikely to ramble.

[-] rimu@piefed.social 6 points 3 months ago

You don't say

[-] Maeve@kbin.earth 1 points 3 months ago

@yogthos@lemmy.ml

[-] yogthos@lemmy.ml 14 points 3 months ago

Yeah, there's a ton of spam now. My view is that devs should use LLMs themselves to scan for issues, and then see if there's anything to fix. But when it comes to accepting reports or patches, you kind of have to be selective. A lot of the time stuff LLMs will flag can be either hallucinated, or not really an issue. A lot of the reports come from automated systems that don't really do any due diligence to figure out if something is an actual issue that needs addressing. So, I can definitely understand why projects might want to stop accepting random bug reports or code submissions going forward.

[-] paraphrand@lemmy.world 7 points 3 months ago

Fully automated systems that file issues sound like a nightmare. I hope it’s easy to ban those as they appear.

[-] yogthos@lemmy.ml 2 points 3 months ago* (last edited 3 months ago)

Yeah, honestly that's the dumbest thing anybody could think of. It's just a pure waste of resources that wastes people's time. Even if these systems find genuine issues, the sheer volume of spam ensures nobody is going to actually look at them.

[-] Maeve@kbin.earth 6 points 3 months ago

I figured you'd probably be sympathetic.

[-] yogthos@lemmy.ml 17 points 3 months ago

The next few years are going to be interesting because we're moving into uncharted territory in a lot of ways. There's a ton of hype around LLMs, and tons of people abusing this tech in every which way, and then there are useful nuggets where people figure out how to apply it effectively. Eventually we'll need to figure out how to suppress the noise and how to start using these things in productive ways.

[+] unemployedclaquer@sopuli.xyz 1 points 3 months ago

[deleted]

[-] spicehoarder@lemmy.zip 0 points 3 months ago

I'll say it till I turn blue in the face. Unless you've been verified as a trusted source, there should be a small donation required for submitting any type of "help" to a project. (pull requests, bug bounties, etc.) Especially since it always requires humans take time out of their day to review the issues and code changes.

If you love the project, donate to it. If you're a trusted source or can't afford even a small donation, get verified.

[-] Swedneck@discuss.tchncs.de 1 points 2 months ago

that entirely depends on the size of the project though, i'm not gonna demand donations for people to contribute to my fucking modpack with 1000 downloads..

[-] spicehoarder@lemmy.zip 1 points 2 months ago

I mean, thats exactly my point. If the donation money wouldn't pull you out of a hole that has you feeling like you're stuck in an inescapable nightmare, this option isn't for you.

Linux.zip