as long as our actions are not significantly harmful to others, then there shouldn’t be any laws forbidding those actions.
But you're not being consistent about what "significantly harmful" even means. Loud noises apparently counts, but only if you want it to. Air pollution doesn't, even if you think it should.
Is OP harmed by someone smoking weed in the middle of nowhere?
To be fair, they specifically said "smoking just smells bad and is really unpleasant to be around in the street", so presumably they only really care about the ban when it's near other people and would be enforceable in the first place. So if no one's around, do what you want, but near other people you shouldn't smoke. That goes along rather neatly with your 'personal freedoms so long as there isn't harmful to others.'