Ah, that I don't know.

I spend a lot of time listening to a "Live" stream and that doesn't seem to have thst problem... but if I listen to a playlist, yeah I have that problem.

However, I often have to stop to speak to someone / get coffee, so that seems to reset the timer so it's not toooo bad for me