1254
Moving in the right direction
(thelemmy.club)
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
this post was submitted on 25 Jul 2026
1254 points (95.5% liked)
Fuck AI
7748 readers
893 users here now
"We did it, Patrick! We made a technological breakthrough!"
A place for all those who loathe AI to discuss things, post articles, and ridicule the AI hype. Proud supporter of working people. And proud booer of SXSW 2024.
AI, in this case, refers to LLMs, GPT technology, and anything listed as "AI" meant to increase market valuations.
founded 2 years ago
MODERATORS
I would say its to protect people, if you ask a LLM for a legal answer and trust it, then you could get into legal trouble when it turns out to be wrong.
Not only are LLMs wrong like ~1/10th of the time (for the more advanced ones) but law is also an area where there is black and white rules, but also there is interpretation of some aspects.
Some things are set in stone and some are about understanding the intention of the law and being able to back that up in a court if it was challenged.
Do you want the public trusting a LLM that says "oh yes you can claim a tax deduction for an Xbox because its for stress relief related to your job" and then you get whacked with a fine for incorrectly filing your tax.
Using a LLM to help your draft an email to your uncle = low risk. Using a LLM to help you decide if you actually need a medical procedure your doctor has reccomended = high risk.
If that was the logic we were using then Wikipedia shouldn't be allowed to have articles on health and law either. WebMD shouldn't have ever been legal either.
Obviously, the benefits to having that information available outweigh the hypothetical negatives. Even with them being incorrect, partial and easy to draw invalid conclusions from, having the information be freely and readily accessible is better for everyone.
I think the distinction is that looking up wikipedia or any other website that appears to have topics of information you can learn from is different from asking a tailored answer to your specific query.
One feels like more like research to improve your understanding, but you know you would have to go to a professional to ask specific questions around your circumstances.
And the other feels like your own 'personal professional' that answers all your little specific questions, but is really just a complex algorithm trying to make you happy and doesnt know if is telling you right or wrong, only that it shouldnt say it doesnt know.
A program that can be very confidently wrong while giving people advice that can mess up their life (healthcare, finiancial, legal advice, etc) could benefit some, but will ruin others make it not worth the risk?
Literally WebMD.
I think there's an assumption that professionals are always correct. They are not. Professionals are also heavily trained to "[try] to make you happy and doesnt know if [they are] telling you right or wrong, only that [they] shouldnt say [they don't] know."
You're also neglecting how bad health care access, and legal etc., is. Especially in countries like the USA where access to healthcare is financially gated and increasing extraction makes "go[ing] to a professional" not possible unless you wait >6 months to go somewhere that's a >3hr commute from where you live. WebMD, Wikipedia and chatbots can help to decide whether the >$1,000 and 7hr professional consultation is worth it, and if so be more informed going in so that the time is better spent.
That being said, poor healthcare can absolutely have worse outcomes than no healthcare. I do believe that is a minority of situations, however I also lack any hard evidence to really back that up. Do you have any?
WebMD is just a website with information pages on health topics, not a program or custom response, is it not?
I think that there is a future that it may be a reasonable first step for people with restricted access barriers, but at this stage its too much of the wild west to be using it for these higher level cases.
But regardless i dont have specific information to present you, i woild argue the amount of people that have committ suicide after interacting with LLMs rather than seeing a mental health professional would be a fairly relevant example though.
I thought the issue you had was with the information itself. Why does it "being a program" matter? That said, it is; "symptom checker" specifically and "page rank" more generally.
And? There seems to be this assumption that "wild West" == bad/dangerous/scary/harmful. The modern Internet is certainly far less of the "wild West" it used to be. I would also argue it is also far more dangerous, harmful and nefarious than the "wild West" web.
Anecdotes aren't useless, but they need to be contextualized. Is it higher than the baseline? Is it greater then those who didn't commit suicide because when the mental health institutions failed them they at least had some sort of backup?
Navigating healthcare while dealing with a mental health issues is a paradox. Is the damage caused greater than the damage prevented from allowing access to a tool that can help lower that barrier to entry?
Looking at general trends I find that the harms outweighing the benefits very hard to believe. Do you have anything non-anecdotal, on that specific point, to back it up?