To be honest it's mostly FUD. Self host email is perfectly doable provided that:
- you don't use your home IP
- you do DNS records correctly
- you do DKIM & DMARC correctly
- you don't send spam
What happened to me over the years was having the subnet my ip was is being associated to spammers, and that was a problem. It took several iterations with blacklists and my provider to get it sorted. That's why I recommend using a reputable service provider that take care of spammers in a timely manner.