It's not that you will necessarily perceive that latency in any meaningful way, but in a competitive gaming setting, a few ms of latency means you react a few ms later, which means your opponents get a few extra ms.

I mean 3ms is very good, and nothing to worry about IMO. Pretty sure Windows isn't any better either. Either way, in a competitive setting, every ms counts.