I assure you, this is not default behavior. I use three different pairs of Bluetooth headphones on my new M4, MacBook Pro, and it does not do this. I’ll dig around and see if I can find the setting which may be stupidly hidden somewhere, and I’ll let you know if I find it. I can definitely understand how this could be extremely annoying.
What brand of headphones are you using?