There is an argument to be made that allowing people with unhealthy desires a safe and harmless outlet, they will be less compelled to go with the harmful option.
And, actually, I kinda want to disagree with the premise too. Even if it was provably true that noone gets hurt if there wasn't porn, you can flip the question; why should it be banned if it doesn't hurt anyone? Do you want to live in a world where anything that's perceived as bad is just outright banned without much thought?