77
ChatGPT o3 found a Linux Kernel vulnerability. "The future" has an 8% success rate, and a 28% chance of false positives. (sean.heelan.io)
submitted 1 year ago* by to c/techtakes@awful.systems
10 comments fedilink hide all child comments
 

This blog post has been reported on and distorted by a lot of tech news sites using it to wax delusional about AI's future role in vulnerability detection.

But they all gloss over the critical bit: in fairly ideal circumstances where the AI was being directed to the vuln, it had only an 8% success rate, and a whopping 28% false positive rate!

you are viewing a single comment's thread
view the rest of the comments
[–] 29 points 1 year ago (1 child)

Additionally, we already have tools like Valgrind that would have uncovered the use-after-free bug.

  • permalink
  • fedilink
  • source
  • hideshow 1 child comment
    •
  • [–] 2 points 1 year ago

    Can’t be assed to read the bs but sometimes the use after free only happens in some rarely executed code path, or only when one branch is executed then later another branch. So you still may need fuzzing to trigger use after free for Valgrind to detect.

  • permalink
  • fedilink
  • source
  • parent