I think that may be the case, I'm on the s and it started happening on like 48hrs in, noticed it gets better if I delete my older saves too. So probably related to the limited hardware.
I do think there must be some memory leak involved though, and seems to be worse when Xbox uses its "suspend" mode.