Everyones initial use of chatbots should be on the thing they are most knowledgable about so they can get an idea of how wrong it can be and how it can be useful but you have to treat it like some eager wet behind the ears intern just did for you.

This is what I call "confidently wrong". If you ask it about things you have no clue about, it seems incredibly well-informed and insightful. Ask it something you know deeply, and you'll easily see it's just babbling and spouting nonsense - sure makes you wonder about those earlier statements it made, doesn't it?

This is where you have to check out the reference links it gives as if they were search results and the less you know the more you have to do it. I mean people have been webMDing for a long time. None of these things allow folks to stop critical thinking. If anything it requires it even more. This was actually one of my things with ai and work. The idea is for it to allow people with less knowledge to do things and to me its kinda the reverse.

Something I found today - ask it for the lyrics of your favorite song/artist. It will make something up based on the combination of the two and maybe a little of what it was trained on... Even really popular songs (I tried a niche one by Angelspit first then tried "Sweet Caroline" for more well known). The model for those tests was Gemma3. It did get two lines of "Sweet Caroline" correct but not the rest.

The new gpt-oss model replies with (paraphrased) "I can't do that because it is copyrighted material" which I have a sneaking suspicion is intentional so there's an excuse for not showing a very wrong answer to people who might start to doubt it's ""intelligence"" when it's very clearly wrong.

... Like they give a flying fuck about copyright.

Its more than that. It takes the input and tries to interpret the bad grammar and sentences into search terms and finds link the correlate the highest to its interpretation and then gives back the response that summarizes the results with good grammar and plausible sentences. Again this is why I stress that you have to evaluate its response and sources. The sources are the real value in any query. Im actually not sure how much the chatbots give sources by default though as I know I have not gotten them and then asked for them and now I get them as a matter of course so im not sure if it learns that I want them or if they did a change to provide them when they had not before.