It's just that the same people can afford to own all those houses because they get them rented out to tourists who pay an exuberant amount of money to stay somewhere for a week or two.

If there would be less tourists, the houses should be rented out, because an unused house is money lost. And if locals can't afford them , and tourists don't use them, the only option is to lower the prices.

And there is a lot of large landowners that would start losing a lot of money fast if they wouldn't manage to rent out their houses. Makes me think of a bubble of sorts.