You're talking about two very different technologies though, but both are confusingly called "AI" by overzealous marketing departments. The basic language recognition and regressive model algorithms they ship today are "Machine Learning", and fairly simple machine learning at that. This is generally the kind of thing we're running on simple CPUs in realtime, so long as the model is optimized and pre-trained. What we're talking about here is a Large Language Model, a form of neural network, the kind of thing that generally brings datacenter GPUs to their knees and generally has hundreds of parameters being processed by tens of thousands of worker neurons in hundreds of sequential layers.
It sounds like they've managed to simplify the network's complexity and have done some tricks with caching while still keeping fair performance and accuracy. Not earth shaking, but a good trick.