however, people are not notoriously bad at the types of basic arithmetic they test for. every time I pay something with cash, I work out how much change I'm owed mentally, and so does the seller. I can count on one hand the number of times I've actually been given incorrect change throughout my entire lifetime. And when I did get wrong change, it was usually "oh, I thought you gave me €10 ínstead of €20". Meaning that they actually still did the math correctly.

No sane person will ever tell you 4 is bigger than 7. Yet llms sometimes get even this type of question wrong. They learn patterns, but not concepts. This is even simpler than basic arithmetic.