I feel like the same kind of argument can probably extend to either intellectual property or real physical objects. With physical objects certain limits have to apply of course (like me withholding things you need to survive could potentially justify your theft).
With intellectual property, if you write stories for yourself to pass the time you aren't obligated to share/sell those stories to me and it would be wrong for me to break into your home and make copies of them if you chose not to sell/share them with me.