216
submitted 4 days ago by BrikoX@lemmy.zip to c/linux@lemmy.zip

Multiple researchers using the same tools to find the same bugs are creating ‘unnecessary pain and pointless work’

all 20 comments
sorted by: hot top new old
[-] lordbritishbusiness@lemmy.world 63 points 4 days ago

I've found that the LLMs tend to over classify and nitpick a fair bit, often missing broader context that accounts for the flaw being tolerated or undiscovered.

They're not wrong, but have no context for triage and so give far too many results. It forces you to consider an LLM subscription yourself just to keep up with the other LLM users which is starting to feel like some form of zero sum red queen's race.

The tsunami of reports won't be receding for a while yet, and we can only hope the teams on the receiving end don't drown in it.

[-] lvxferre@mander.xyz 36 points 4 days ago* (last edited 4 days ago)

Reminder: if you can put code in a chatbot and get it to find bugs, the devs can do it too. As such, even if your "LLM bug finding trip" works, it's still useless, and a waste of everyone else's time.

Is it really a waste of time if the computer can find the bugs, and the dev can focus on fixing them instead of pulling double duty?

[-] lvxferre@mander.xyz 6 points 4 days ago

It is because everyone else's computer is finding that same bug, and they're all reporting it separately, and reduplicating efforts associated with trialling the bugs. It's clearly more costly than having the dev themself fire a spare machine and have their computer alone find bugs.

[-] luciferofastora@feddit.org 2 points 3 days ago

The human will still need to check and understand the bug.

Particularly if a given bug is reported dozens of times in slight variations, someone will have to check for each of these reports whether it's a bug already reported earlier. If they're all checked by the same person, that person may quickly recognise "Okay, yeah, that's the same thing I logged earlier" but if it's multiple people, there's just so much extra overhead associated with keeping track of what's new and what's "30th reported of same bug".

The AI generating the report probably doesn't check if the issue hss been found before. If the people subvmitting it also don't, you end up with a load jof chaff by lazy people thinking they're helpful when really they're obstructing the efforts.

[-] Lon3star@lemmy.world 36 points 4 days ago

Welcome to AI

[-] Jankatarch@lemmy.world 21 points 4 days ago* (last edited 4 days ago)

It must suck because all chatbot output I seen speaks like a corporate email, just can't get to the point.

[-] Swedneck@discuss.tchncs.de 2 points 9 hours ago

one of the big reasons i don't understand how people can use anything other than basic search assist stuff, that's at least directly rephrasing stuff like wikipedia and thus unlikely to ramble.

[-] spicehoarder@lemmy.zip 5 points 4 days ago

Seriously, I can't stand talking to Chat GPT

[-] rimu@piefed.social 7 points 4 days ago

You don't say

[-] Maeve@kbin.earth 2 points 4 days ago
[-] yogthos@lemmy.ml 14 points 4 days ago

Yeah, there's a ton of spam now. My view is that devs should use LLMs themselves to scan for issues, and then see if there's anything to fix. But when it comes to accepting reports or patches, you kind of have to be selective. A lot of the time stuff LLMs will flag can be either hallucinated, or not really an issue. A lot of the reports come from automated systems that don't really do any due diligence to figure out if something is an actual issue that needs addressing. So, I can definitely understand why projects might want to stop accepting random bug reports or code submissions going forward.

[-] paraphrand@lemmy.world 7 points 4 days ago

Fully automated systems that file issues sound like a nightmare. I hope it’s easy to ban those as they appear.

[-] yogthos@lemmy.ml 2 points 4 days ago* (last edited 4 days ago)

Yeah, honestly that's the dumbest thing anybody could think of. It's just a pure waste of resources that wastes people's time. Even if these systems find genuine issues, the sheer volume of spam ensures nobody is going to actually look at them.

[-] Maeve@kbin.earth 6 points 4 days ago

I figured you'd probably be sympathetic.

[-] yogthos@lemmy.ml 17 points 4 days ago

The next few years are going to be interesting because we're moving into uncharted territory in a lot of ways. There's a ton of hype around LLMs, and tons of people abusing this tech in every which way, and then there are useful nuggets where people figure out how to apply it effectively. Eventually we'll need to figure out how to suppress the noise and how to start using these things in productive ways.

[-] spicehoarder@lemmy.zip 1 points 4 days ago

I'll say it till I turn blue in the face. Unless you've been verified as a trusted source, there should be a small donation required for submitting any type of "help" to a project. (pull requests, bug bounties, etc.) Especially since it always requires humans take time out of their day to review the issues and code changes.

If you love the project, donate to it. If you're a trusted source or can't afford even a small donation, get verified.

[-] Swedneck@discuss.tchncs.de 1 points 9 hours ago

that entirely depends on the size of the project though, i'm not gonna demand donations for people to contribute to my fucking modpack with 1000 downloads..

[-] spicehoarder@lemmy.zip 1 points 6 hours ago

I mean, thats exactly my point. If the donation money wouldn't pull you out of a hole that has you feeling like you're stuck in an inescapable nightmare, this option isn't for you.

this post was submitted on 18 May 2026
216 points (99.5% liked)

Linux.zip

903 readers
4 users here now

Linux community for Lemmy.zip. ~~also this needs mods pretty bad~~ apparently not as bad as I thought (either that or this community isnt alive enough for troublemakers yet)


Community Rules:

  1. Do not violate any laws, third-party rights, and/or proprietary rights.
  2. Do not harass others, be abusive, threatening, and/or harmful.
  3. Do not be needlessly defamatory and/or intentionally misleading.
  4. Do not upload without marking obscene and/or sensitive content as such.
  5. Do not promote racism, bigotry, hatred, harm, and violence of any kind.

^i^ ^may^ ^or^ ^may^ ^not^ ^have^ ^stolen^ ^these^ ^rules^ ^from^ ^another^ ^linux^ ^community^ ^on^ ^another^ ^instance^

founded 2 years ago
MODERATORS