963
Peer reviewing is torture now (thelemmy.club)
submitted 2 days ago by to c/programmer_humor@programming.dev
139 comments fedilink hide all child comments
 
you are viewing a single comment's thread
view the rest of the comments
[–] 3 points 2 days ago (8 children)

I've seen too as I have to review it. AI produces very professionally looking bad code.

It is so weird to explain, but that's basically what I see.

Before LLM I could immediately tell someone's code is crap, now I have to spend a lot of time trying to understand what it does to reach the same conclusion.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 8 child comments
    •
  • [–] 0 points 2 days ago (7 children)

    My honest opinion is that it's bad because a lot of people using LLMs have no standards and push the first thing that seems to work. Be mad at who's at the driving wheel, not the car.

    You absolutely can generate crap with agents/LLMs, and like a humans writing, the first draft will probably be subpar or maybe complete garbage. Every new session is a clean slate, that's why putting effort in the documents guiding it is so important.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 7 child comments
    •
  • [–] 3 points 1 day ago (4 children)

    The car analogy doesn't work because you're not the one making the code/driving the car. It drives itself, and you sometimes can propose some directions for it to steer.
    The problems with llm aren't just that sometimes your shit doesn't work or obviously bad, that's what you're talking about, and that's what minimally responsible sloperators can catch. The main problem is it writes something that looks OK to a human (that's a criteria for it) but sometimes it's unexpectedly idiotic in random places (because not being idiotic wasn't a criteria). You need to check way more thoroughly for it, and you can't use normal shortcuts that help you with it. So you obviously don't do that.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [–] 1 point 1 day ago (3 children)

    See, that's the issue. People letting it "drive itself" get worse results. You should be the one holding the standards and guiding the model, otherwise you will be frustrated.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 3 child comments
    •
  • [–] 2 points 1 day ago (1 child)

    I saw this advice too, the problem with being very detailed is a question why not use a less ambiguous languages to do it?

    Why not write the explanation in language like Python. Why should I write it in English when I can say the same thing in Python and it is actually easier to do for me.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 1 point 1 day ago*

    This kind of specification applies to the agent behavior or the agent behavior in a project. You don't write it on every conversation, you write/review it once for the project or agent and let the harness include it on every conversation. You can even ask the agent to scan the codebase and pull desirable patterns out of it for future sessions.

    The entire point of using an agent is to not have to write everything yourself, so that when you write "implement feature X", the feature gets implemented in a way that makes sense in that project, is reviewed, refactored, and tested by agents, and it's good enough to bring in a human reviewer.

  • permalink
  • fedilink
  • source
  • parent
  • [–] 2 points 1 day ago (1 child)

    Maybe I have some ridiculously high standard, but whenever I used it, I wasn't happy with what it generated.

    I noticed that the time it did it + the time for me to review it and possibly fixed was at best the same amount of time it took me to write, at worst it was longer.

    For hard problems that I stumbled on it was useless.

    For simple problems it worked and produced good code, but it still took less time for me to write it than waiting for Claude to finish thinking.

    Everyone swears that they can produce good code with LLM and it is others who are bad, and LLM is force multiplier to them. But from what I see the only time it can speed up their work is if they never review it or even take time to understand the actual problem they are trying to solve.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 2 points 1 day ago*

    I don't know you, but it doesn't sound like a high standards issue to me, sounds like a lack of process. I've been a thorough reviewer before AI, at least thorough in the ways that mattered, not nitpicking formatting. And I'll tell you an automated reviewer today can catch more things before I have time to confirm the first item I find. There are still false positives, but it's still more thorough than I have time to be. Bc of that it really helps including a round of automated review before looping in a human, regardless of who/what wrote the code.

    The main kind of review problems AI still struggles with are the project direction ones: "does it make sense to implement this/like this?", "should this be a new package instead?", and things involving tacit knowledge that often goes undocumented "last time we did this, someone had to access prod on a Sunday" - so that's what I focus my reviews on. And the other area is if you're writing UI code, whether it's a web app or a game, it'll also struggle to determine what "feels" good to use, so it'll need a human earlier in the loop.

  • permalink
  • fedilink
  • source
  • parent