Perhaps they track who you talk to and show you ads that are relevant to those people, or their best guess based on two profiles.
I don't think there's a data center out there with a live audio stream of literally billions of always-on devices 24/7/365.
Perhaps there's some local processing first, but devices have permissions for apps, and lights that indicate the mic/camera is in use.
I figure someone would have figured it out by now (reverse engineering, decompiling code), or someone from Google/Apple/Samsung would have leaked it if it were true. Think of the number of people required to keep this secret.