I have friends with kids and are in happy living relationships, but they're definitely not upset to get some away time from the chaos of young children when they go to school or daycare. That's not to say they don't love them or love being around them, but it's a lot of work and a break isn't inherently a negative thing.
I don't like boomer humour like making constant insults about one's in laws or their spouse. I do think that your coworker definitely came with the wrong tone and probably fits that bill. Solid in you for shutting that shit down.