You:

they have it set up so it will use tools like Me:

It farms arithmetic out to dedicated, non-language based sub agents

Is there an echo in here?

Even if I interpret you charitably, you are using the word "it" to mean "LLMs", instead of "AI". That isn't valid or useful. Nobody cares very much whether specific classes of neural network are good at maths; they care whether ChatGPT is going to give them the right answer. Go waste some of OpenAI's money right now by asking it a question on mathematics, and see whether "AI can do maths".

This gets even more ludicrous when you look at what AI is enabling in real mathematics, rather than in arithmetic.

It can do it better than I thought. Normally it'd spin Python, as you said, so I tried asking it to not use tools: