It might not work, but it would serve to increase awareness that you absolutely cannot trust these machines to give you correct answers, ever.

The nice thing about laws is that they don't give a shit. If they say you can't do something, it's your problem to make sure you don't do it.

Anything that requires a license is a good start.

Some legislation is to lay the ground work to expand it later.

Easier to lay a sub floor and put carpet over it later, than to just put carpet over the studs. When you're changing a massive thing like this, it's better to start small and work outwards. Canada introduced universal Dental and Phamacare and it was limited to certain demographics. People bitched about "Well what about me?"

You're coming. Start with what/who is most important, make sure the system works, and carry on from there.

New York State not New York City.

I don’t know why Mamdani is pictured

the text in this image is also just flat-out wrong: the proposed legislation does not actually "ban AI from answering questions" on these subjects at all, as this post claims it does.

https://www.nysenate.gov/newsroom/press-releases/2026/kristen-gonzalez/ny-state-senator-kristen-gonzalez-her-bill-address-ai explains that her bill is strictly about impersonating licensed professionals:

We have documented instances of AI chatbots providing false medical license numbers to users and documented cases of real harm from AI chatbots providing misinformation. This is what my common-sense legislation addresses—protecting users from misinformation, scams, and fraud. This legislation does not prohibit a user from asking a chatbot questions or receiving general information and advice, as long as the chatbot is not presenting that information as a licensed professional. It would also not prohibit licensed professionals from consulting AI technology in the course of their work.

Using the internet or using a chatbot? Big difference here.

I diagnosed myself

using the internet

I'm going to assume you researched medical documents, studies, and journals to link things together, and not ask a chat bot "My foot hurts, what is wrong?"

You're smart enough to self diagnose yourself with readily available information to the public, but not smart enough to not know the difference between that and a chat bot. That's remarkable.

I diagnosed myself

Congrats, glad your life is hopefully changing for the better because of this.

I hate taking the wind out of your sails but this is in the area of anecdotal evidence. You are a single person saying this for your specific case. I am sure we will get another few people confirming the same here.

But what about how many people tried this and got nothing useful out? Or even a misdiagnose? Anyone enthusiastic about this possibility of AI diagnosing people wouldn't really mention this. People like to mention the successes but it needs to be weighed against the failures too, and I don't think there is a good record of that.

If there is, its probably a medical study and I'd venture a guess that AI did not perform as well as a trained doctor.

I agree. You're spot on. This isn't a technical problem. We need to think less like engineers and more like lawyers on these things.

You don't require AI companies to have their models refuse to give out legal advice. As you note, that's a fool's errand. Instead, you make it a felony to use any LLM to write legal advice. Currently lawyers are already facing court sanctions for using AI to write their briefs. If you really want to get serious, you make it a felony. Apply the same penalty to a doctor using an LLM to diagnose their patients as you would if I decided to open a clinic in my house and start practicing medicine without a license. Charge people operating AI therapists with practicing psychology without a license.

This isn't a technical problem, it's a legal one. You don't try to regulate the LLMs themselves. You just make it a severe felony to use that technology in certain highly sensitive applications and professions.

And sure, one could say that we don't have a reliable way of screening AI from human made work. But again, that would be thinking like an engineer, not a lawyer. You don't enforce these laws by screening thousands of legal briefs or medical records with an AI detection filter. You offer huge cash rewards to whistle blowers, so that a disgruntled employee can rat out their bastard boss that's forcing everyone to illegally use AI. Apply massive fines and give half of whatever is collected to the person who blows the whistle.

Are you a structural engineering company owner forcing your employees to use AI to engineer buildings? Now you have to worry about one of those employees, an employee who probably really dislikes using AI, ratting you out. And their whistleblower reward will be so large, they'll end up owning the company. They won't have to worry about being blacklisted from future employers, as they'll be getting enough money to retire.

Don't look for technical solutions for the problem of AI abuse. Look for human solutions.

When people say "AI" these days, they mean the LLM-derived generative AI such as ChatGPT, Google Gemini, Anthropic Claude, and Grok.

These AI systems are not capable of finding authoritative search results. They are good at generating fluent, plausible, context-appropriate text. In legal and medical situations, that's the opposite of what you'd want.

Do you know how often doctors will take a patients history, step outside the exam room, and then google shit?

Think about how often the doctors land on some pseudo-science result. Now imagine that instead of a doctor it's a patient and instead of asking "could these be related" the patient is told "these are related".

a big ol disclaimer that it’s NOT to be taken seriously

This has worked very well for Fox News, which is not a news agency and is for entertainment purpose only. Also for dietary supplements, which are not medicine.

I'd agree with you to some extent if people weren't so god damn stupid. So many lazy fucks just take the AI's output at face value, believing it to be some all knowledgeable god, rather than a compilation of all human biases expressed on the internet with a bunch of hallucinations thrown in for good measure.

I'm guessing this isnt about saving jobs, it's about reducing the harm caused by people taking advice from a misinformation machine for complex issues like the law and their physical and mental health.

Sorta agree. Maybe force them to have a very obvious sign that says AI is routinely wrong about these topics specifically and are not a source to consider reliable, and then also offer a helpline where one can get reasonable answers even if the answer is "unfortunately you need to seek expert opinion on this"

But googling basic labor laws and simple healthcare tricks should be acceptable like stretches for wrist or knee pain, skin/hair care or what level of diarrhea is hospital worthy should be fine.