Some of the things you brought up Google does appear to be doing with their AI. Unfortunately I think this is a damned if you do damned if you don't scenario. AI (especially integrated with a security nightmare system like IoT) isn't trustworthy in it's own right, but becomes a security nightmare of epic proportions when you realize that most IoT devices that connect to the Internet don't even have rudimentary security protocols. When you add the fact that Gemini is trying to do IoT and at the same time do things like call screening, text recognition etc (for the purposes of spam removal and scam mitigation), and that to do so it has to collect and monitor your data to do it, you recognize why so many security people are distrustful.
Google says they don't collect that data. That the processing is done ok device rather than requiring to be sent back to Google for processing. They say that this data won't be used to further train the AI. People don't trust it