Yes it will affect you even if you use Firefox. If a lot of us still used Firefox, Google would not be able to do it as websites would not give up on a big chunk of their audience.