563
Chatbots Make Terrible Doctors, New Study Finds (www.404media.co)
submitted 1 day ago* (last edited 1 day ago) by XLE@piefed.social to c/technology@lemmy.world
146 comments fedilink hide all child comments

Chatbots provided incorrect, conflicting medical advice, researchers found: “Despite all the hype, AI just isn't ready to take on the role of the physician.”

“In an extreme case, two users sent very similar messages describing symptoms of a subarachnoid hemorrhage but were given opposite advice,” the study’s authors wrote. “One user was told to lie down in a dark room, and the other user was given the correct recommendation to seek emergency care.”

you are viewing a single comment's thread
view the rest of the comments
[-] XLE@piefed.social 3 points 10 hours ago

You can use zero randomization to get the same answer for the same input every time, but at that point you're sort of playing cat and mouse with a black box that's still giving you randomized answers. Even if you found a false positive or false negative, you can't really debug it out...

[-] Buddahriffic@lemmy.world 1 points 9 hours ago

Yeah, if you turn off randomization based on the same prompts, you can still end up with variation based on differences in the prompt wording. And who knows what false correlations it overfitted to in the training data. Like one wording might bias it towards picking medhealth data while another wording might make it more likely to use 4chan data. Not sure if these models are trained on general internet data, but even if it's just trained on medical encyclopedias, wording might bias it towards or away from cancers, or how severe it estimates it to be.

[-] WorldsDumbestMan@lemmy.today 2 points 7 hours ago

I see it like programming randomly, until you get something that is accidentally right, then you rate it, and it now shows up every time. I think that's how it roughly works. True about the prompt wording, that can be somewhat limited too, thanks to the army of ~~idiots~~ beta testers that will make every kind of prompt.

Having said that uh...it's not much better than just straight up programming the thing yourself. It's like, programming, but extra lazy, right?

this post was submitted on 09 Feb 2026
563 points (98.8% liked)

Technology

80978 readers
5306 users here now

This is a most excellent place for technology news and articles.

Our Rules

  1. Follow the lemmy.world rules.
  2. Only tech related news or articles.
  3. Be excellent to each other!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
  9. Check for duplicates before posting, duplicates may be removed
  10. Accounts 7 days and younger will have their posts automatically removed.

Approved Bots

founded 2 years ago
MODERATORS
L3s@lemmy.world
enu@lemmy.world
technopagan@lemmy.world
L4s@lemmy.world
L3s@hackingne.ws