That's why you always use discipline in handling security credentials. Two factors won't save you if your lack of discipline gets both of them compromised.

And I don't appreciate other people's lack of discipline creating risks for me. Password databases and private keys can be backed up, but if I lose my phone for some reason, I also lose anything that depended on that phone for authentication, and I have no way to recover quickly from such an event.