Also, a key part of how GPT-based LLMs work today is they get the entire context window as their input all at once. Where as a human has to listen/read a word at a time and remember the start of the conversation on their own.

I have a theory that this is one of the reasons LLMs don't understand the progression of time.