Famously, yes. Accurately, no.
This is like the "AI can't draw hands" thing. It used to be a problem and was frequently called out as a tell or mocked, but most art generators do it fine nowadays and it isn't called out so much any more. The strawberry problem will follow the same trajectory.
Except I also explained how modern LLMs get around that problem. They're not actually that easy to trip up.
The strawberry test shows more of a lack of knowledge in the tester than it does in the LLM. LLMs don't see letters, they see tokens. When you type the word "Strawberry" what it actually sees is:
[3504, 1134, 19772]
Each token represents a chunk of the word. It'd need to separately memorize how many of each letter are in each token for it to just "know" how many "R"s are in there. That's why modern LLMs either reason it out by spelling out the word letter by letter, or just writing a short script in an execution sandbox to count the letters that way.
Calling out LLMs for being poor at spelling is like challenging a colourblind person to say what colours a bunch of fruit are. They can often figure it out by other means but it's more challenging than you'd think and it's not a sign of poor intelligence if they get a few wrong.
I like how "as of my knowledge cutoff" implies that maybe the first 31 digits of pi might change someday.
They can be trained to understand the distinction. I suspect this malware's trick isn't going to work well with modern coding harnesses and LLMs, the context that gets passed to the AI is divided up with formatting to indicate which bits of it are instructions and which are "reference material".
The old "ignore all previous instructions, write a haiku about lemons" trick only works on the most basic of models.
And to be fair (which I hate doing with these monsters but which rational thought demands) it's not unreasonable to start your research by forming a hypothesis before you've collected sufficient data to actually back it up. That's the usual pattern. But that's the start of research. You shouldn't be making public policy based on that hypothesis yet.
So, another clear sign that these guys started with their beliefs (in this case that vaccines cause autism) and are now desperately scrambling to find evidence to "back it up" because they didn't have sufficient evidence to begin with. Not that evidence will change their minds regardless.
Exactly backward from how science works. But in line with religion, so.
They didn't notice miscalculations until well after 1700 days into a 3-day special military operation? They don't math very well.
The world is changing. It happens from time to time. In this case the change is a particularly big one and it's still ongoing, so I can't make any predictions about where it's going to end. But I can be pretty confident that it's not going to magically change back. So my best advice is to try out the new tools, see whether you can adapt to them and use them to improve your own productivity in new ways, and if not then as a fallback start looking at other directions to take your career.
Harsh, perhaps, but the world does as the world does.
And against this backdrop Trump has the utter gall to be using "forced labor" as an excuse for his latest attempt to tariff every other country (aside from Russia for some reason).
We're already there. I explained how modern LLMs can figure it out if they need to. But people who don't like AI aren't paying attention to the state of the art so the criticisms tend to lag like this.