Wasps generally won't fuck with you, but here's a pro tip.

It's been shown that they can recognize and remember human faces. No idea how they have enough wetware for that kind of pattern recognition, but whatever.

So, do not look at them! And for damn sure don't wave your arms about. If they think you're any sort of threat, it's game on.

Had a nest over my front door for months and they didn't hassle me or my pig, and we went in and out that door frequently. Pointedly ignoring them worked, but I finally nuked the nest so none of my friends would get stung.