Sure, you can agree to anything.
If you didn't think it through and thus suffer from skill issues.
And there are of course good stories to tell with it, like in this secret traitor situation, and good players will apply circumstantial bonuses fairly.
Like perhaps that paladin doesn't WANT to believe their comrade is a murderer.
Doesn't mean you shouldn't be aware that another player can't force you into simple contested rolls on the nature of reality that you can't possibly contest, ever.
Hell, even if they're right! You can play a completely deluded madman that looks at a windmill, hears an NPC tell him the absolute, objective truth that it is a windmill, and decides it's a giant instead.