Here's a post from someone who also doesn't like the Turing Test. As they point out, you can pedantically call it a Turing Test but it's a version that was very deliberately rigged in favor of the AI, including the tests only being ~4-5 exchanges, which is completely ridiculous for trying to make a thorough evaluation by this metric. I don't think it has all that much to do with gullibility because the limitations of these models become much more apparent over time. It's just more headline-mill bullshit. I don't share the author's view that the "coaching" is a relevant factor to consider the outcome's validity, though.

Granted, I'm also not trying to say that the Turing test is the ultimate metric or anything, just that it's an extremely low baseline that, employed in good faith, current LLMs plainly do not clear. They often can't even pass for one prompt if the one prompt is "spell strawberry" or something like that.

Edit: I also think the alternative that they propose is not great because it's mostly a question of video-processing. It's getting too hung up on information-processing questions to use something other than text.