From both sides when it bugs ya is a pain for sure. Not even weed, if I had 2 chain smokers as my neighbors I'd be very annoyed.
Obviously it'd be better for you if they were smoking less or were somehow able to direct it away from your place, but maybe a couple small / medium hepa filters will at least ease your discomfort instead of having to keep windows closed when you maybe want them open?