I'm sure they would only venture onto you if they were persuing something that already ran onto you

You're big and scary and dangerous to something as small as a dinner plate sized spider, they're not going to risk walking on megafauna (did you know we are megafauna?)

Tiny spiders might not have the perspective to recognise that we're living things, but they crawl on us when we're awake. Huntsmans keep a distance from us, and think they're perfectly camouflaged while they stand still, as they are when they're not on an ivory white wall