JackFrostNCola comments on Moving in the right direction

[–] JackFrostNCola@aussie.zone 5 points 2 months ago (7 children)

I would say its to protect people, if you ask a LLM for a legal answer and trust it, then you could get into legal trouble when it turns out to be wrong.
Not only are LLMs wrong like ~1/10th of the time (for the more advanced ones) but law is also an area where there is black and white rules, but also there is interpretation of some aspects.
Some things are set in stone and some are about understanding the intention of the law and being able to back that up in a court if it was challenged.

Do you want the public trusting a LLM that says "oh yes you can claim a tax deduction for an Xbox because its for stress relief related to your job" and then you get whacked with a fine for incorrectly filing your tax.

Using a LLM to help your draft an email to your uncle = low risk. Using a LLM to help you decide if you actually need a medical procedure your doctor has reccomended = high risk.

permalink

fedilink

source

parent

hideshow 7 child comments

[–] SinAdjetivos@lemmy.world 3 points 2 months ago (6 children)

If that was the logic we were using then Wikipedia shouldn't be allowed to have articles on health and law either. WebMD shouldn't have ever been legal either.

Obviously, the benefits to having that information available outweigh the hypothetical negatives. Even with them being incorrect, partial and easy to draw invalid conclusions from, having the information be freely and readily accessible is better for everyone.

permalink

fedilink

source

parent

hideshow 6 child comments

[–] JackFrostNCola@aussie.zone 1 point 2 months ago (5 children)

I think the distinction is that looking up wikipedia or any other website that appears to have topics of information you can learn from is different from asking a tailored answer to your specific query.

One feels like more like research to improve your understanding, but you know you would have to go to a professional to ask specific questions around your circumstances.
And the other feels like your own 'personal professional' that answers all your little specific questions, but is really just a complex algorithm trying to make you happy and doesnt know if is telling you right or wrong, only that it shouldnt say it doesnt know.

A program that can be very confidently wrong while giving people advice that can mess up their life (healthcare, finiancial, legal advice, etc) could benefit some, but will ruin others make it not worth the risk?

permalink

fedilink

source

parent

hideshow 5 child comments

[–] SinAdjetivos@lemmy.world 1 point 2 months ago (4 children)

A program that can be very confidently wrong while giving people advice that can mess up their life

Literally WebMD.

I think there's an assumption that professionals are always correct. They are not. Professionals are also heavily trained to "[try] to make you happy and doesnt know if [they are] telling you right or wrong, only that [they] shouldnt say [they don't] know."

You're also neglecting how bad health care access, and legal etc., is. Especially in countries like the USA where access to healthcare is financially gated and increasing extraction makes "go[ing] to a professional" not possible unless you wait >6 months to go somewhere that's a >3hr commute from where you live. WebMD, Wikipedia and chatbots can help to decide whether the >$1,000 and 7hr professional consultation is worth it, and if so be more informed going in so that the time is better spent.

That being said, poor healthcare can absolutely have worse outcomes than no healthcare. I do believe that is a minority of situations, however I also lack any hard evidence to really back that up. Do you have any?

permalink

fedilink

source

parent

hideshow 4 child comments

[–] JackFrostNCola@aussie.zone 1 point 2 months ago (3 children)

WebMD is just a website with information pages on health topics, not a program or custom response, is it not?

I think that there is a future that it may be a reasonable first step for people with restricted access barriers, but at this stage its too much of the wild west to be using it for these higher level cases.

But regardless i dont have specific information to present you, i woild argue the amount of people that have committ suicide after interacting with LLMs rather than seeing a mental health professional would be a fairly relevant example though.

permalink

fedilink

source

parent

hideshow 3 child comments

[–] SinAdjetivos@lemmy.world 1 point 2 months ago (2 children)

I thought the issue you had was with the information itself. Why does it "being a program" matter? That said, it is; "symptom checker" specifically and "page rank" more generally.

its too much of the wild west

And? There seems to be this assumption that "wild West" == bad/dangerous/scary/harmful. The modern Internet is certainly far less of the "wild West" it used to be. I would also argue it is also far more dangerous, harmful and nefarious than the "wild West" web.

the amount of people that have committ suicide after interacting with LLMs rather than seeing a mental health professional would be a fairly relevant example though.

Anecdotes aren't useless, but they need to be contextualized. Is it higher than the baseline? Is it greater then those who didn't commit suicide because when the mental health institutions failed them they at least had some sort of backup?

Navigating healthcare while dealing with a mental health issues is a paradox. Is the damage caused greater than the damage prevented from allowing access to a tool that can help lower that barrier to entry?

Looking at general trends I find that the harms outweighing the benefits very hard to believe. Do you have anything non-anecdotal, on that specific point, to back it up?

permalink

fedilink

source

parent

hideshow 2 child comments

[–] JackFrostNCola@aussie.zone 1 point 2 months ago (1 child)

Look mate, all i am saying is that cutting edge technology shouldnt be used for critical areas until it is proven safe, i really am not invested enough in the cenversation to continue this thread, i support the descision being made to restrict LLM use for these areas at this point in time.

LLMs have been linked to mental health impacts, specifically demonstrably in suicide cases (see: https://en.wikipedia.org/wiki/Deaths_linked_to_chatbots) and i do believe in a lot of these cases the LLMs caused greater harm than would have been presented without the LLM interference, and definitely less harm than if these people saw a human professional to talk about their issues rather than a program. Yes there is a cost of seeing a health professional but this is not a case of 'somethings better than nothing'.

permalink

fedilink

source

parent

hideshow 1 child comment

[–] SinAdjetivos@lemmy.world 1 point 2 months ago

I do think that helps clear it up for me and helps me understand the disconnect we're having. If you're done don't let this pressure you into responding, but I'm responding for the sake of trying to resolve the difference in perspective.

We agree that there are real harms from this technology. We agree that human professionals are, on average, significantly better especially within the context of healthcare.

We disagree on what "safe" means. I am coming from the perspective that there is no such thing. There will always be side effects irregardless of the domain, technology, intervention, etc. and so the question is does it do more harm than good?

I do not see a single instance among that list, or others, that isn't just a blatant failure of existing healthcare and social systems. Without ascribing supernatural social powers to this technology I fail to see how the LLM usage is anything more than a symptom of those institutional failures.

Legislating away access will do nothing more than give the monopoly power to the same institutions responsible for the initial failure and the initial conditions that led to that point while providing them with a scapegoat.

permalink

fedilink

source

parent