tux snipes-hesitation

you are viewing a single comment's thread
view the rest of the comments
[–] 9 points 3 hours ago (4 children)

genuine question from me, the uninformed: If a change or commit is being presented as code a person authored (or without indicating one way or another) how can someone tell it is LLM slop vs code a person wrote?

Like, with writing, there are sort of semi-intangible patterns that make me recognize AI slop, but even then I don’t know for sure- its mostly based on suspicion that comes from reading AI slop in the first place. Like one example I can think of is the “Not X, but Y” summarization at the end of nearly any essay or brief explanation of something. Its always meant to be poetic or impactful and it might have been a genuine style at one point -I think it was common in advertisements maybe?- but now it makes me think, “huh, this is probably slop”

So for code, is there some BS that they are always doing that feels inhuman or wrong? Do they make the same errors or add unnecessary shit that makes people roll their eyes? Or is it just a high volume of work with only a short turnover? If someone asked an LLM to write a fix to a bug they encountered, and then submitted it as their own fix, how does one know it is slop?

  • source
  • hideshow 4 child comments
  • [–] 15 points 2 hours ago

    The Linux kernel gets a ton of PRs constantly. My big issue with LLM code is that it is really good at appearing to be thought out, but actually reverse engineering it can reveal major mistakes.

    A good reviewer should still catch them, but it takes longer since the "code smell" is different. Humans will usually make consistent simple mistakes while LLMs will sometimes use advanced patterns to make the same mistake.

    The fact that these take longer to review, are usually much longer in general, and are appearing at an incredible pace (1hr to generate, 1 day to review) creates a backlog that becomes unbearable to review.

    A lot of open source projects have banned them for this reason, but the ones that do allow them tend to use LLM reviewers to help clear the backlog which is a whole different ouroboros of an issue.

  • source
  • parent
  • [–] 9 points 2 hours ago

    It occurs eventually once the slop enjoying programmer is de-skilled where they only glance at AI output and start losing track of the bigger picture.

    The most obvious one for code is the "comment above a line of code that describes exactly what the code does"

    # This does A with B as an input
    DoA(B)
    
  • source
  • parent
  • [–] 10 points 2 hours ago

    afaik LLM code is usually unnecessarily verbose and tries to include libraries that typically wouldn’t be necessary.

    But also it seems that after writing work emails, writing code seems to be the thing LLMs do best.

  • source
  • parent
  • [–] 2 points 1 hour ago

    Yes, there is a flavor to LLM code. Generally there is poor structure and comprehensibility, the more code slides right off your brain when you read it despite there being lots of comments the more likely it is to be slop. On the other hand, bad human written code tends to have a feeling of impenetrability that's often accompanied by a lack of comments.

  • source
  • parent