People can be toxic, downvotes are just a tool to indicate relevance or not to the community theme, though most people subvert that and use as a popularity contest regardless.

I tend to ~~upvote~~ (edit: downvote) less often than once a month, but occasionally do find it useful - e.g. if someone has -10 downvotes and at least one person has already responded to indicate why, there is little point to adding another reply, but another downvote adds weight to the countervailing POV.

Note that hexbear.net has downvotes disabled, yet is widely regarded by the entire Fediverse as the literal #1 most toxic place here by far. Instead of downvoting, people are forced to reply - and quite often use things like images of pig butts (literally) in the process of pooping.

Note that PieFed has an option for each individual community to decide whether it wants to enable or disable downvoting - it would be nice to see this in Lemmy too? Most often I think downvotes have a real, actual purpose to serve, but rarely I could see cases where they do not.

Usually I think the problem is something else - like people coming in from All, not bothering to read the rules, yet adding their opinions regardless. e.g. people who feel the need to man-splain into women's communities. In such cases it's not the downvotes per se that are bad but the input from non-community members. Btw PieFed has an option for a community mod to set that restriction as well.

In most cases like this, the limitation is simply that Lemmy is too primitive and needs to add features to allow nuance and subtlety, but its feature development is slow as molasses. (Then again, PieFed has its own... issues as well.) Maybe more people will contribute code to help speed it along? Unfortunately donations won't do much - a great deal of finances goes to moderation of the Lemmy.ml instance to help spread its political ideology, and Dessalines has refused to allow donations to exclusively speed up code development but not be used for the latter purpose. So, we make do with the tools we have as best we can.

I am not in favor of downvotes being removed, but some instances do that already so move your account there if that is what you personally want for yourself. The top example there is probably Reddthat.com.