The bigots use the language to conflate trans women and cis men, to suggest that they're really the same. They use that language to shape the understanding of people who don't know any better. And those people absolutely impact your life, because it's those people voting for transphobia to "keep women safe" because they see you and I as being just as dangerous as cis men, because language like that shapes their understanding.