Too me one of the big issues is being able to trust a government or business to not trace a person's identity back through the token. There are technical ways to prevent that as far as I'm aware, but there's such a strong incentive against such protections that it's really hard to trust unless you're technologically skilled enough to verify the process yourself.