I've definitely had both. Sometimes the hosts of the actual podcast read an ad using their own voices. In this case everyone gets the same audio file and crowdsourcing the timestamps would work.
For dynamically inserted ads, it will be more complicated. Maybe a system like content id that has a library of known ads and detects them in the audio.