Yeah, that's why there's all the variations of pushing an extremely large person in front of the train to stop it, and things like that. The lever, obviously it's a choice that you should make. The person, it's still a choice, but at what point is it not an issue you should try to handle.
Voting is a lever. There's other actions that are more akin to pushing someone onto the tracks.