Only problem I can see with that is, that passive scores take away from the randomness attributed to DnD but I generally agree with you. I also don't like rolling checks for my players.