We live in a capitalist society. Money equals power. Giving a place your money effectively contributes to their power. Unfortunately that means it's not entirely unreasonable to equate patronage with support and it's generally reasonable to draw moral conclusions based on what people support.

Obviously it shouldn't be the only or primary way we engage in activism, but boycotting or shifting patronage is a reasonable part of a political movement under capitalism. Sometimes patronage cannot be helped, but we should do what we can to support places (sometimes financially) that best align with our morals.