Tbf there's plenty reasons other than "robbing you" that people wouldn't like cops, personally mine is "I constantly smell like weed which is illegal here and I don't want to jeopardize my freedom," for some others it may be "I'm black," etc.
That said they can't get mad if you're unwilling to do the deal in another highly public place. If I asked "can we do it at the nearby gas station" and you said "no" I'd maybe think you're overly cautious but that's your right, I'm being overly cautious too about not getting arrested for smoking weed so I get it, "ok good luck, not for me then."