1249
Moving in the right direction
(thelemmy.club)
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
this post was submitted on 25 Jul 2026
1249 points (95.7% liked)
Fuck AI
7738 readers
2455 users here now
"We did it, Patrick! We made a technological breakthrough!"
A place for all those who loathe AI to discuss things, post articles, and ridicule the AI hype. Proud supporter of working people. And proud booer of SXSW 2024.
AI, in this case, refers to LLMs, GPT technology, and anything listed as "AI" meant to increase market valuations.
founded 2 years ago
MODERATORS
If they come to the wrong conclusion at worst they go to the hospital and someone there tells they are wrong and not to worry. We haven't ban googling your symptoms.
You've missed the point.
Someone who Googled their symptoms and ended up at WebMD were presented with reliable albeit ambiguous information. Carefully curated by medical professionals and not random snake oil salesmen or homeopathy gurus. They fail to interpret the symptoms correctly, because of course they would. If they are looking up causes of a headache, it can be anything from brain tumor to low blood sugar, or even no reason at all.
Someone who asks AI for medical advice is going to get the entire internet worth of fucking opinions, as well as medical journals and such, because that's where ChatGPT, Gemini, Grok and Claude all got their data from. And it will string those words together in a way that sounds authoritative, but it will have literally zero concept of what it's talking about. It's an overgrown autocomplete. You ask for it to provide some citations, but no amount of asking for citations will compel people to actually read them, nor will there be any guarantee that the LLMs couldn't just hallucinate that as well.
It's like taking a Google search for medical information and adding a middleman who occasionally filters in junk data, then wrapping it in a nice package, giving the impression of authority and accuracy, making the process as frictionless as possible.
There can be no reasonable expectation that people who have not historically been capable of making good choices in regards to seeking online medical advice will suddenly start doing it all of their own volition.
You might say "What's the harm if they end up at the hospital either way"? But the danger is that they never go to the hospital at all because the chatbot told them to sun their perineum or inject bleach with the confident prose of a medical doctor and they just... do it. Because why would the AI be wrong?
And who is to blame when someone is injured or killed as a result? Doctors have standards, and they take an oath to first do no harm. If they make a mistake, they are liable for it. How can you hold a machine intelligence accountable for inaccurate diagnoses?
The simple answer is you can't, and so logically they should be strictly prohibited from masquerading as any kind of authority on medicine.
If LLM answers had zero concept they wouldn't work. We are here talking about it because the answers are correct up to a certain point that makes them good enough.
These middleman are also called teachers, when you go to medical school you have many of these, you don't get handled the medical encyclopedia and left on your own.
that's something you are making up, an LLM telling you to inject bleach would be the same as it telling you that 2+2= coconut, it's the wrong answer, if all the answers where wrong we wouldn't be here talking about this tech.
Doctors prescribe you medicines and do surgery on you, an LLM can't do these things, if AI tells you to take a medicine you still have to go to the medic to get these.
The propaganda meme is talking about a ban on AI answers about medicine. LLMs have no authority on medicine, that's something we will be discussing in 20 years.