to me there is no harm in it.

Well yeah, but you could say the same about hitting a pedestrian with your car. To you, there's no harm. But the person you're cheating on, the person who's trust you've violated, who's time you've wasted, who feels stupid and used and humiliated after the relationship is over, there's a lot of harm for them.