Definitely been done, though. There are lots of fantasy worlds where divine beings are objective fact, which is often directly experienced by protagonists. AFAIK in those cases, the claim by in-universe atheists tends to be that those are just powerful beings, not gods, which kind of seems like a distinction without a difference to me.

Though it does become philosophical. What IS a "god", and if we accept these beings as gods, should you worship them? Are they worship-worthy, and are they going to punish people who refuse to worship them? The last part is kind of independent of whether they're really gods, even a powerful being who is literally just a mundane human (e.g. a king) can still make your life hell if you refuse their demands.