I see the irony. I suppose two differences are that, one, the survey was optional, and two, although participants might not necessarily know who was reading the answers, the answers weren't (I don't think) generally accessible.
Voting data is not generally accessible, either, but it appears to be accessible to admins of other instances. If this is the case, it wouldn't be difficult to set up an instance, make a post through any account about a specific topic, and observing the data to see who up/downvotes it. This could narrow down the list of people that a bad actor might want to target for further data harvesting.
Considering how many billions are put into surveillance, data collection, and controlling (social) media, I don't think we should discount this fear as farfetched. Even if, on its own, it's fairly benign data.
I've also noticed that some of the new instances/users are a lot happier with 'local' communities. If this grows into, say, a list of 'things to do in my area' communities, it wouldn't take much for a fascist to identify local targets by a series of what I'm going to call 'voting traps'.
Maybe I'm paranoid.