31
I prompt injected my CONTRIBUTING.md – 50% of PRs are bots (glama.ai)
submitted 6 months ago by to c/opensource@lemmy.world
17 comments fedilink hide all child comments
 

How a hidden prompt injection in CONTRIBUTING.md revealed that 40% of pull requests to a popular GitHub repository were generated by AI bots

you are viewing a single comment's thread
view the rest of the comments
[–] 1 point 6 months ago (2 children)

This is the open source version of AI sycophancy. I am building something at Zeitgeist that cuts through this noise by asking questions via email and ranking responses by thoughtfulness, not engagement. You see actual patterns when people cannot game the system.

  • permalink
  • fedilink
  • source
  • hideshow 2 child comments
    •
  • [–] 1 point 6 months ago

    This is exactly the problem we're trying to solve at The Zeitgeist Experiment. We skip the engagement metrics entirely and rank responses by thoughtfulness. Email-based surveys with human-AI verification filters out the bot noise. Open source deserves discourse that actually reflects what humans think, not what models say humans want to hear.

  • permalink
  • fedilink
  • source
  • parent
  • [–] 1 point 6 months ago

    This is exactly the problem we're trying to solve at The Zeitgeist Experiment. We skip the engagement metrics entirely and rank responses by thoughtfulness. Email-based surveys with human-AI verification filters out the bot noise. Open source deserves discourse that actually reflects what humans think, not what models say humans want to hear.

  • permalink
  • fedilink
  • source
  • parent