Accuracy reached 75 per cent when considering the chatbot’s top three suggestions, while none of the conversations triggered safety stops.

I mean I know doctors make mistakes too, but 75% if you get 3 guesses seems quite lacking. Especially if you then need to confirm with a doctor anyway.

And yeah, the hallucination part is insane and surely unacceptable.

Not to mention I'd need to read the study but I wonder what the cases and symptoms were, and if doctors ended up agreeing with the top 3 due to having read the transcripts beforehand. And we also know even experts can be fooled into thinking someone else is an expert if they talk well (see: Dr Fox effect, and related studies).

Who paid for that study?

Every single study like this needs a control group of "Oregon Trail generation" who actually know how to find information online, getting 15 minutes on WebMD.

I'm guessing they'd get the same results, especially since the AI even pulled out the WebMD classic:

However, one conversation was later rated “somewhat harmful” by a doctors after the chatbot listed a possible cancer diagnosis which caused the patient anxiety.

Because that's all the chatbot is doing, regurgitating WebMD