Let's start with this. Google Assistant, Siri, Cortina Alexa, etc are always listening.
Your phone can be listening, but at least on Android the microphone permissions are necessary, and unless there are native exploits I'm super unaware of, this happens at the lowest levels.
Keep your phone in your pocket, check the app permissions, learn how to check data leaving your network, and learn to love Big Brother.