Indirect becomes nebulous. At what degree of indirect harm do we set that limit. Almost every action we do may cause indirect harm to others. It might be better phrases as "physically" harms someone. I don't want to get into someone doing something to themselves like taking drugs and restrict it solely on the basis that it will hurt their family and friends to see what happens to them.
I use it as the core base of my beliefs, but that doesn't mean I don't think that freedom divests them of any responsibility for their indirect actions. It's the default position until something convinces me why it should be restricted or outlawed.
I also limit it to individuals working alone. Once they work in groups and organize the damage that can be done is different. Or doing it for commercial reasons. I believe private businesses can only exist under strict regulation.