I am anti-theist, and fuck no to banning public displays of anything. It's in the name - public. Public space belongs to everyone. Freedom of expression should not be a privilege restricted to people who can afford to buy or rent a place to exercise it.
If you can prove harm, we can ban the harm. Any and all bans must be tightly focused on restricting only harm and to a greater degree than it inherently restricts freedom. Elsewise, we're just oppressing dissent/diversity and essentially abandoning freedom itself as a core value. And the fact that we're talking about dictating what people can do on or with their own bodies raises the stakes that much higher. Seriously, this is a dangerous path and the hazards far greater than any possible reward.
Tax religion. Remove their privilege. Do not create a new underclass.