Furbag comments on Moving in the right direction

[–] Furbag@pawb.social 3 points 2 months ago (1 child)

You've missed the point.

Someone who Googled their symptoms and ended up at WebMD were presented with reliable albeit ambiguous information. Carefully curated by medical professionals and not random snake oil salesmen or homeopathy gurus. They fail to interpret the symptoms correctly, because of course they would. If they are looking up causes of a headache, it can be anything from brain tumor to low blood sugar, or even no reason at all.

Someone who asks AI for medical advice is going to get the entire internet worth of fucking opinions, as well as medical journals and such, because that's where ChatGPT, Gemini, Grok and Claude all got their data from. And it will string those words together in a way that sounds authoritative, but it will have literally zero concept of what it's talking about. It's an overgrown autocomplete. You ask for it to provide some citations, but no amount of asking for citations will compel people to actually read them, nor will there be any guarantee that the LLMs couldn't just hallucinate that as well.

It's like taking a Google search for medical information and adding a middleman who occasionally filters in junk data, then wrapping it in a nice package, giving the impression of authority and accuracy, making the process as frictionless as possible.

There can be no reasonable expectation that people who have not historically been capable of making good choices in regards to seeking online medical advice will suddenly start doing it all of their own volition.

You might say "What's the harm if they end up at the hospital either way"? But the danger is that they never go to the hospital at all because the chatbot told them to sun their perineum or inject bleach with the confident prose of a medical doctor and they just... do it. Because why would the AI be wrong?

And who is to blame when someone is injured or killed as a result? Doctors have standards, and they take an oath to first do no harm. If they make a mistake, they are liable for it. How can you hold a machine intelligence accountable for inaccurate diagnoses?

The simple answer is you can't, and so logically they should be strictly prohibited from masquerading as any kind of authority on medicine.

permalink

fedilink

source

parent

hideshow 1 child comment

[–] brachiosaurus@mander.xyz 1 point 2 months ago

but it will have literally zero concept of what it’s talking about.

If LLM answers had zero concept they wouldn't work. We are here talking about it because the answers are correct up to a certain point that makes them good enough.

It’s like taking a Google search for medical information and adding a middleman who occasionally filters in junk data, then wrapping it in a nice package, giving the impression of authority and accuracy, making the process as frictionless as possible.

These middleman are also called teachers, when you go to medical school you have many of these, you don't get handled the medical encyclopedia and left on your own.

You might say “What’s the harm if they end up at the hospital either way”? But the danger is that they never go to the hospital at all because the chatbot told them to sun their perineum or inject bleach with the confident prose of a medical doctor and they just…

that's something you are making up, an LLM telling you to inject bleach would be the same as it telling you that 2+2= coconut, it's the wrong answer, if all the answers where wrong we wouldn't be here talking about this tech.

And who is to blame when someone is injured or killed as a result? Doctors have standards, and they take an oath to first do no harm. If they make a mistake, they are liable for it. How can you hold a machine intelligence accountable for inaccurate diagnoses?

Doctors prescribe you medicines and do surgery on you, an LLM can't do these things, if AI tells you to take a medicine you still have to go to the medic to get these.

The simple answer is you can’t, and so logically they should be strictly prohibited from masquerading as any kind of authority on medicine.

The propaganda meme is talking about a ban on AI answers about medicine. LLMs have no authority on medicine, that's something we will be discussing in 20 years.

permalink

fedilink

source

parent