The oft-forgotten full phrase is: "a broken clock is right twice a day, but you still wouldn't use it to tell time."

Similarly, often in reference to corrupt cops or politicians: "one bad apple spoils the whole bunch."