I was part of a project that tried. If you just want to get rid of dialog trees local processing might be an option, but for an actual reactive world your looking at possibly hundreds of dollars a month in rented compute. The deeper problem is the current generation of LLMs are just too unstable over time, and frankly, too dumb to maintain its persona and keep up with a dynamic fictional world. Each hallucination amplifies the disconnect from the fictional reality, and trying to reset it changes the presentation of the persona. And if the humans are actively trying to mess with the AIs or if the AIs are supposed to consider something without blurring it out, its all going to crash and burn.

Of course, its a little concerning that these agents are too unreliable to be a character is a fictional video game, but apparently reliable enough to be given access and authority in the real world.

I agree, and it's very possible. I think the real limiter is that consoles don't have the RAM necessary to load the models to enable it. Maybe in the future, when models become more efficient in memory usage, or more likely, the RAM shortage is resolved.

I dont know how realistic my dreams are there.

Yeah, as I said in another comment in this post, I don't think LLMs can do anything like that, and I think what you're describing is going to require something that doesn't exist yet and that we have no reason to think will exist any time soon. (And I'm super pissed that Nvidia et al are raking in billions in dirty grifter money on the contrary promise -- which it can't keep, mind you.)