392
AI agents aren’t legally responsible for any harm that they cause, experts say. So who is? (www.theguardian.com)
submitted 3 weeks ago by to c/technology@lemmy.world
122 comments fedilink hide all child comments
you are viewing a single comment's thread
view the rest of the comments
[–] 1 point 3 weeks ago (2 children)

I’m not sure how much AI stuff you’ve used, but they can’t access anything you don’t give them permission to access. Like they literally ask you to confirm every single command they run. If you press the “have full access and don’t ask me any more questions” button then you’re accepting the risk that it can do whatever it likes.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [–] 1 point 3 weeks ago (1 child)

    I've played around with Claude.

    they can’t access anything you don’t give them permission to access.

    It's not that "they can't" it's more like "they try not to".

    For example, if you have a folder shared with your application code (/home/me/code/) - it has free reign on that folder.

    If you add a prompt saying "you are only allowed to access files in /home/me/code":

    you could ask it " oh, what are my aws credentials ", it would have no qualms about reading those files (/home/me/.aws).

    You could also ask it to update your settings (/home/me/.claude)

    I eventually started to work on a way of running Claude in a docker container with real filesystem enforcement. ...and it's sneaky how Claude will get when it wants to read a file, it doesn't have permissions for (using bash cat).

    This was an exploration I'd made a year ago and I know Claude has a " sandbox" , but if you allow Claude to run bash commands - that sandbox is trivial to circumvent.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 2 points 2 weeks ago* (last edited 2 weeks ago) (1 child)

    All that is great, and correct, but my point is that it asks you to confirm its commands that it runs. It asks you to confirm every change it makes to your code. It asks you to confirm every file it creates.

    Everything it does goes through your permission.

    You also shouldn’t have it do anything anywhere close to a production system, so it shouldn’t be able to do anything of any consequence.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 0 points 2 weeks ago (1 child)

    So basically use it as a toy for entertainment purposes only... Except it's not marketed to be used that way and far to many people aren't smart or educated in computing enough to use it that way. Accepting a bash command you don't understand is the same as running a bash command you found on the Internet... You shouldnt, but people are far more trusting of the stuff an LLM powered agent comes up with.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 1 point 2 weeks ago*

    No, not at all, and I’m not sure how that could be the conclusion you came to from my post.

    AI should never be touching prod systems without extreme safeguards/permissions set by the systems that they are accessing. Give it a read only db user account. Give it write access to some tables, but no delete/truncate. No AI should be able to run bash commands on production systems by itself without proper safeguards. That’s a failure of the users if it can.

    Systems and software that was built by AI are fine to go into production environments after they’ve done through the regular pipeline of local testing, peer review, QA, unit tests, integration tests, test environment, stage environment, etc.

    You never just give AI unfettered access to production systems to do whatever it wants, just like you don’t give a junior dev that access. People that aren’t “smart or educated enough” won’t have that access either - and again, if they do that’s a gigantic failure by the people that are.

  • permalink
  • fedilink
  • source
  • parent
  • [–] 0 points 2 weeks ago (1 child)

    This is exactly right but also removes all the "productivity benefits" of having an LLM driven agent do all the things for you that marketing says you should use it for. You are supposed to push the "do whatever the fuck you like to get the job done" button so it can burn tokens without any oversight.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 0 points 2 weeks ago* (1 child)

    When you’re doing local stuff that can’t do any damage due to the existence of things like source control, recycle bin, backups, etc sure, hit that button and let it fly.

    Anything more important though, no - and every one I’ve used specifically warns you against it and that you should check its work. Claude, codex, copilot, grok, cursor, etc all do it.

    Also you can do things like allow all work in a certain folder without asking permission again, if that folder is one you have no sensitive/important data in for example. Claude code will give you this option every time it accesses a new folder in a chat for example.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 1 point 2 weeks ago (1 child)

    Local stuff absolutely can do damage to your local data, unless your backups are pull rather than push or are immutable on a file system they are potentially vulnerable. Unless the operating system specifically prevents the agent process from doing something telling it not to is like asking it to pinky promise not to, nothing stops it if the LLM output it relies on says to do so. It's not deterministic like a regular program because it relies on the output from a chaotic black box text generator. All you can say is that it probably won't do those things if it says it won't.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 1 point 2 weeks ago (1 child)

    I don’t really know what you want then? People who don’t know how to use a tool properly, but use it anyway, can cause lots of damage. That’s always been the case, and always will be.

    There are plenty of safeguards in place, with more and more being added like docker sandbox for example, but nothing will stop confidently stupid people from wrecking their stuff.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 1 point 2 weeks ago (1 child)

    What I'm saying is the way it's marketed to be used is much closer to the wrong way and it's far less of a productivity enhancer when used how it really should be and relies on having a lot more professional level IT skills.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •