Lay it on me. Maybe I'll convince myself that I'm a bot after all. Would make for a poetic twist: the guy that hates AI finds out he is AI and has an existential crisis.
More seriously, that does raise a good question. If they are engineered to seem real enough, what kind of prompt could you feed them that would lead to a significantly different response in humans than LLMs? Just how do we have to construct Turing Tests in this era?