Your eyes wander. They don't just look directly ahead. Directly ahead is not the only thing you're looking at. That's just not how we work. We're constantly scanning around looking at different objects. Sure, DoF accurately simulates looking at exactly where you're facing, but that doesn't accurately simulate the way we look at our environment.