a sufficiently capable fakery which still has limitations can still upend the world quite a bit
Maybe but we are essentially throwing petabyte sized models and lots of compute power at it and the results are somewhere on the level where a three year old would do better in not giving away that they don't understand what they are talking about.
Don't get me wrong, LLMs and the other recent developments in generative AI models are very impressive but it is becoming increasingly clear that the approach is maybe barely useful if we throw about as many computing resources at it as we can afford, severely limiting its potential applications. And even at that level the results are still so bad that you essentially can't trust anything that falls out.
This is very far from being sufficient to fake AGI and has absolutely nothing to do with real AGI.