You can have it off and just use the "song search" on the top menu, but yeah. I assume it just keeps itself activated even if you disable it, they just try to be less blatant about catering to you.
Anecdotally, I've only had lime 2 instances of "google was definitely listening to show me this ad" in the 6 years I've had pixels, and I've seen just as many instances with other phone types. You truly have to go out of your way to avoid the surveillance.