238
Fed up with vibe coders, dev sneaks data-nuking prompt injection into their code (arstechnica.com)
submitted 2 months ago by to c/programming@programming.dev
46 comments fedilink hide all child comments
you are viewing a single comment's thread
view the rest of the comments
[–] -2 points 2 months ago* (4 children)

How to get yourself blacklisted by large sweeps of the FOSS community:

Step 1: Include any kind of undocumented subversive behaviour in your thing.

That's it, doesn't matter what the intent is, simply by demonstrating you are willing to include anything that is remotely subversive without being open about it is usually enough to get blacklisted by a lot of people, because if you did it once... who's to say you won't do it again, but possibly worse next time?

People are extremely coldly receptive to anytime a FOSS dev throws a sudden undisclosed anything in their tool, let alone one that is actively malicious.

If I'm gonna depend on work life on anything FOSS, I ain't touching anything like that, regardless of intent, with a 200 foot pole lol.

All it takes is one button click to get notified:

  • permalink
  • fedilink
  • source
  • hideshow 8 child comments
    •
  • [–] 20 points 2 months ago (1 child)

    any kind of undocumented subversive behaviour in your thing.

    Fortunately, this behaviour is explicitly documented.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] -2 points 2 months ago (2 children)

    They only documented it after all the outcry, which is way too late.

    Documenting it post release still counts as having released undocumented behavior.

    And if its malicious (which this 100% is), then it doesn't fuckin matter anyways lol. You now are treated akin to a trojan maintainer by companies. You'll get flagged as "don't ever use anything by this person"

    Super great way to get yourself flagged and lose any opportunity in the future for possibly licensing stuff you maintain for big bucks. What company would risk paying money to someone who does childish stuff like that lol

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [–] 15 points 2 months ago (2 children)

    imo it's more accurate to call it polarizing and get you blacklisted by the types of people you maybe don't want using your code anyways. Personally anyone doing this I'm going to be more likely to use their code

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [–] 5 points 2 months ago (2 children)

    I understand the sentiment, if you don’t like AI code generation you’re probably thinking you’re on the same side. But what happens if this person finds something else they hate that you don’t hate, and finds a way to sabotage that? They’ve already demonstrated a willingness to be destructive. And you’re running their code so they don’t need anything even remotely as dumb as some AI agents to exploit, they can just write destructive code normally.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [–] 7 points 2 months ago

    Is it merely hating AI code generation or is it "AI code generation is in practice anti-FOSS" (unless there's an ethical AI out there, trained exclusively on public domain code, that I don't know about)?

  • permalink
  • fedilink
  • source
  • parent
  • [–] 4 points 2 months ago (1 child)

    You can decide if you want to use it or not, at your own risk. It's free software, written by people in their free time, they owe you nothing.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 0 points 2 months ago* (1 child)

    Sure, you have that right.

    And companies will exercise that right by blanket blacklisting everything related to you which can have huge sweeping impacts on your career lol

    Its a super super stupid move to make. You are free to do a lotta other shit that tanks your career too lol

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] -1 points 2 months ago

    by the types of people you maybe don’t want using your code anyways

    ...companies? Sure I guess, if you want to angle your career trajectory towards "unemployable" by all means lol.

    Personally anyone doing this I’m going to be more likely to use their code

    I am a tech lead, if any dev under me intentionally added/used a tool to our systems because it had malicious undocumented behaviors of any kind, they would be fired immediately and any company that contacted us for reference would be informed of their behavior.

    To be clear, this is the scenario of

    Me: hey I saw you installed [tool], that thing is flagged by our systems for the maintainers having done malicious undocumented stuff in the past

    Dev: haha yeah thats why I used it

    Me: you are joking right?

    Thatd be an instant high level escalation to "strip this person of privs and get them off our system asap, and HR now has to be involved"

    You dont fuckin do shit like that in a real company if you wanna stay employed lol.

  • permalink
  • fedilink
  • source
  • parent
  • [–] 6 points 2 months ago* (last edited 2 months ago) (1 child)

    Most open source maintainers never "license [any] stuff you maintain for big bucks" that is often hard to do and/or goes against the philosophy of open source entirely.

    And I don't even think this is malicious behaviour as it just nukes the code of this package and nothing else if you are not being careful yourself...

    If you don't do version control you are not a good programmer, imo

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] -1 points 2 months ago

    Most open source maintainers never “license [any] stuff you maintain for big bucks” that is often hard to do and/or goes against the philosophy of open source entirely.

    Uhhh... no this is actually very common. Usually with scaling licenses, "free for use if your company is below [threshold]", its super common...

    And I don’t even think this is malicious behaviour as it just nukes the code of this package and nothing else if you are not being careful yourself…

    Are you even reading what you just wrote lol.

    Being "sorta" malicious is still malicious. And companies usually have zero tolerance for that shit.

    If you don’t do version control you are not a good programmer, imo

    You really underestimate how much damage this could do then, lol...

  • permalink
  • fedilink
  • source
  • parent
  • [–] 2 points 2 months ago (1 child)

    it's not subversive. it's a string, it has no effect on the code output. Only a rogue bot would interpret it as anything except a string. No human user would ever encounter an issue.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 0 points 2 months ago

    And a line of code that deletes system32 is only meant to be interpreted by the CPU, not a human, too.

    Whats your point?

    Injected attacks that instruct an AI to perform a malicious action dont get absolved just because the attack vector us through an agent.

  • permalink
  • fedilink
  • source
  • parent