Ah. Free association is what it'd be called by libertarians. I see what you mean now.

Was he against government-forced segregation?

I'm kind of bad in these situations (from a libertarian perspective) because I tend to refuse to worry about things I like the outcome of.

If you and I had incredibly similar views on how people should behave, and we both put time and money into achieving those outcomes, but I didn't support using violence (libertarian reductionist view of government), does that mean I support things I'm ostensibly fighting personally?