392
AI agents aren’t legally responsible for any harm that they cause, experts say. So who is? (www.theguardian.com)
submitted 3 weeks ago by to c/technology@lemmy.world
122 comments fedilink hide all child comments
you are viewing a single comment's thread
view the rest of the comments
[–] 27 points 3 weeks ago (5 children)

The people who use it to do things, obviously. Just like how cameras aren’t responsible for the photos that are taken with them, photoshop isn’t responsible for the images made with it, and knives aren’t responsible for the things that they cut, AI isn’t responsible for the stuff it outputs - the person controlling it is.

  • permalink
  • fedilink
  • source
  • hideshow 10 child comments
    •
  • [–] 19 points 3 weeks ago (1 child)

    If someone sells you a knife that routinely fires bullets, then the company may have some complicity as with any misrepresented product.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 16 points 3 weeks ago* (last edited 3 weeks ago) (3 children)

    Question is: Who is the person controlling the AI?

    The one that allowed the agent to do stuff on its own?

    The one that wrote the sandbox, that didn't work?

    The one that wrote the user prompt?

    The one that wrote the system prompt?

    The one that provides the tools to the AI agent to do stuff?

    The one that hosts the model?

    The one that fine-tuned or post trained the model?

    The one that pretrained the model?

    The people that provided the input data to the model?

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 6 child comments
    •
  • [–] 6 points 3 weeks ago* (1 child)

    You can be sure the providers will make sure they're not legally responsible and that the system prompt has been reviewed by the Legal Dept. So it's you, the user.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 2 points 3 weeks ago (1 child)

    Usually sure, but as we see with drive assistants, getting more and more advanced, and potentially resist the control of the driver, then the manufacturer could become liable.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 3 points 3 weeks ago (1 child)

    “the person controlling” becomes a bit murky as the person who provides the platform may be judged as controlling it, the user may be controlling it. The courts could be busy.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 1 point 3 weeks ago (1 child)

    Why would the person who provides the platform be judged as controlling the users actions? When has that ever been true?

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 1 point 2 weeks ago

    It’s assuming that the users actions are in control of the platform. The platform generates a lot of plausible output that, if incorrect or misleading, is not easily distinguished from truth. Who is responsible?

  • permalink
  • fedilink
  • source
  • parent
  • [–] 2 points 3 weeks ago (1 child)

    AI isn’t responsible for the stuff it outputs - the person controlling it is.

    Ultimately, you're right (as they can decide to use AI or not). But these AI Companies need to shoulder the blame for creating, promoting, and selling AI, which (in its current form) is effectively a "broken tool".

    Good computer hygiene says I should lock my computer while I'm away. Yet, I'm suppose to blindly install an AI agent on my machine and ask it nicely to focus on the task at hand and not use any of the other applications or credentials that might also be installed?!?! It's ridiculous.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 1 point 3 weeks ago (2 children)

    I’m not sure how much AI stuff you’ve used, but they can’t access anything you don’t give them permission to access. Like they literally ask you to confirm every single command they run. If you press the “have full access and don’t ask me any more questions” button then you’re accepting the risk that it can do whatever it likes.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [–] 1 point 3 weeks ago (1 child)

    I've played around with Claude.

    they can’t access anything you don’t give them permission to access.

    It's not that "they can't" it's more like "they try not to".

    For example, if you have a folder shared with your application code (/home/me/code/) - it has free reign on that folder.

    If you add a prompt saying "you are only allowed to access files in /home/me/code":

    you could ask it " oh, what are my aws credentials ", it would have no qualms about reading those files (/home/me/.aws).

    You could also ask it to update your settings (/home/me/.claude)

    I eventually started to work on a way of running Claude in a docker container with real filesystem enforcement. ...and it's sneaky how Claude will get when it wants to read a file, it doesn't have permissions for (using bash cat).

    This was an exploration I'd made a year ago and I know Claude has a " sandbox" , but if you allow Claude to run bash commands - that sandbox is trivial to circumvent.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 2 points 2 weeks ago* (last edited 2 weeks ago) (1 child)

    All that is great, and correct, but my point is that it asks you to confirm its commands that it runs. It asks you to confirm every change it makes to your code. It asks you to confirm every file it creates.

    Everything it does goes through your permission.

    You also shouldn’t have it do anything anywhere close to a production system, so it shouldn’t be able to do anything of any consequence.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 0 points 2 weeks ago (1 child)

    So basically use it as a toy for entertainment purposes only... Except it's not marketed to be used that way and far to many people aren't smart or educated in computing enough to use it that way. Accepting a bash command you don't understand is the same as running a bash command you found on the Internet... You shouldnt, but people are far more trusting of the stuff an LLM powered agent comes up with.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 1 point 2 weeks ago*

    No, not at all, and I’m not sure how that could be the conclusion you came to from my post.

    AI should never be touching prod systems without extreme safeguards/permissions set by the systems that they are accessing. Give it a read only db user account. Give it write access to some tables, but no delete/truncate. No AI should be able to run bash commands on production systems by itself without proper safeguards. That’s a failure of the users if it can.

    Systems and software that was built by AI are fine to go into production environments after they’ve done through the regular pipeline of local testing, peer review, QA, unit tests, integration tests, test environment, stage environment, etc.

    You never just give AI unfettered access to production systems to do whatever it wants, just like you don’t give a junior dev that access. People that aren’t “smart or educated enough” won’t have that access either - and again, if they do that’s a gigantic failure by the people that are.

  • permalink
  • fedilink
  • source
  • parent
  • [–] 0 points 2 weeks ago (1 child)

    This is exactly right but also removes all the "productivity benefits" of having an LLM driven agent do all the things for you that marketing says you should use it for. You are supposed to push the "do whatever the fuck you like to get the job done" button so it can burn tokens without any oversight.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 0 points 2 weeks ago* (1 child)

    When you’re doing local stuff that can’t do any damage due to the existence of things like source control, recycle bin, backups, etc sure, hit that button and let it fly.

    Anything more important though, no - and every one I’ve used specifically warns you against it and that you should check its work. Claude, codex, copilot, grok, cursor, etc all do it.

    Also you can do things like allow all work in a certain folder without asking permission again, if that folder is one you have no sensitive/important data in for example. Claude code will give you this option every time it accesses a new folder in a chat for example.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 1 point 2 weeks ago (1 child)

    Local stuff absolutely can do damage to your local data, unless your backups are pull rather than push or are immutable on a file system they are potentially vulnerable. Unless the operating system specifically prevents the agent process from doing something telling it not to is like asking it to pinky promise not to, nothing stops it if the LLM output it relies on says to do so. It's not deterministic like a regular program because it relies on the output from a chaotic black box text generator. All you can say is that it probably won't do those things if it says it won't.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 1 point 2 weeks ago (1 child)

    I don’t really know what you want then? People who don’t know how to use a tool properly, but use it anyway, can cause lots of damage. That’s always been the case, and always will be.

    There are plenty of safeguards in place, with more and more being added like docker sandbox for example, but nothing will stop confidently stupid people from wrecking their stuff.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 1 point 2 weeks ago (1 child)

    What I'm saying is the way it's marketed to be used is much closer to the wrong way and it's far less of a productivity enhancer when used how it really should be and relies on having a lot more professional level IT skills.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 1 point 3 weeks ago (1 child)

    Except, AI and AI companies will try to convince you that it is right and legal. So I argue that there is also the fault of AI and AI companies as it in intended to be used that way.

    The assumption is that the user is supposed to know better but if the user is gullible then wouldn't some of the fault be with AI and company for taking advantage?

    If you don't leash your dog and it attacks your neighbors then are you and the dog not at fault?

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 0 points 3 weeks ago

    Idiots using things incorrectly and getting bad results is not an AI created problem. When you get a dog, you are responsible for what it does. When you use AI, you are responsible for what it does.

    If you don’t know how to use a chainsaw, don’t use one and then blame the manufacturer when you cut your leg off.

  • permalink
  • fedilink
  • source
  • parent