I only ever downvote rudeness, active spam, or something I think is off-topic. I explain off-topic downvotes but don't feel DOWNLOAD MY SCAM APP TODAY http://wwe.scamlink.fakesite.com/ deserves an explanation and just needs to be pushed down in visibility + reported. I am too wary of getting into an online fight with the person I downvoted and reported for hurling slurs or being a snarky sarcastic namecalling ass to someone being genuine lest they start coming after me too, so I don't reply to them and explain "I downvoted you because I think you're being unnecessarily unkind." Half the time I do actually try to reply in a way that I think gently calls them out and tries to defuse, only to delete immediately after replying in fear of conflict—that's like 75% of my deleted comments.
Of course caveats apply, I remember but cannot find this image with an angry person capslocking in understandable disagreement with Hitler saying "but that's just my civil opinion that we should gas all the jews, i do not get why you're so emotional,"
and sealioning is a thing, but I see way more snark towards people engaging in good faith than I see sealions or awful people who actually deserve it. (And part of why I delete, in case I think it's unnecessary rudeness but it is actually a Hitler tone policing the anti-Hitler guy type situation and I just don't realize it.)