If the sex worker is consenting without duress and is being treated well (I recognize that's a big 'if') then I'm fine with it. I have no inherent objections to sex work itself so it would feel like a double standard to judge the people who use it.

Unfortunately the moral waters are muddied by the rampant trafficking, drug abuse, etc. within the industry.