Why would you hold an election, particularly if he’s doing a good job of it?

Well, that's up for debate and should be decided by the people. As you said: It's a slippery slope and I'll add the way to hell is paved with good intentions.