No, sorry, that still doesn't answer my question.
Cosmically controlled goblins are doing the same thing as bandits, but the bandits made the choice to do evil things and the goblins didn't get a chance to refuse. Surely, the people choosing to do evil are worse than those forced to do evil, right? So why are bandits better than goblins?
The suggestions you gave fall kinda flat to me, really. No matter what the in-universe reason is, the DM made the universe. "It's what my character would do" doesn't excuse bad behaviour, and neither does "it's what my gods decided." You're the one who made them do that. You're the one that decided an entire culture of thinking, feeling people are born objectively evil and can be killed en masse. And that's fucked up.