How do you know when you're letting through a valid access, an unnecessary one that could be a vulnerability, and an actively malicious one?
I don't think anyone is saying throw out all access control, they're just saying SELinux adds too much unproductive friction for everyday usage. You said it takes 15m to troubleshoot. But that's not a one time thing, that's 15m that scales with the amount of new programs and updates you're running. And 90% of people aren't even going to be able to tell they're looking at a malicious access if they're in the habit of always working around blocks that show up.