‘management started quietly asking us to write more of it ourselves again’

https://www.youtube.com/watch?v=8JA8i0Cks6Y&list=UU9rJrMVgcXTfa8xuMnbhAEA - video
https://pivottoai.libsyn.com/20261111-but-the-ai-is-super-good-at-updating-code-well-no-it-isnt - podcast

time: 6 min 03 sec

you are viewing a single comment's thread
view the rest of the comments
[–] 3 points 3 days ago

There is a benchmark out there that hints at this phenomenon: https://www.scbench.ai/ The idea is to track how well the model modifies code over multiple change requests. No surprise for readers here that the LLM will bitrot your code quickly, and the odds of it doing so is worse than a coin toss.

  • source