You are counting your saved person twice, once as a 100% save and again when you consider a net count of 4.
You have a 75% chance of saving 1 person, and 25% of killing 4, which means you are still killing 0.25*4 - 0.75*1 = 0.25 people.
Now the logic of "only 25% of bad outcome" is the real gambler's fallacy here. How far would you push it ? Would you pull the lever if it was a 20% chance of killing 10 ? A 10% chance of killing 50 ?
At some number people seem to go "what are the odds it happens to ME" and happily pull a lever even if the consequences outweigh the odds.