When landlords "invest" in the housing market, they are not making the system of providing housing for people better or more efficient. They are buying up a limited supply of properties that exist in desirable areas and then charging people for the right to use them plus a nice profit for themselves. This reduces the supply for a necessary good and drives up prices, making it profitable for landlords, and a massive, efficiency draining, example of rent-seeking for the system as a whole since the landlord's basically don't work and instead take a cut of what everyone else makes doing useful work.
If you invest in some predatory companies you might be investing in companies that do that, or might do some other predatory practice, but you can also just be putting money into a business so that it has more money to grow its operations, or invest in some new efficiency that makes them run better, and that then both returns a profit back to both of you and helps improve the system as a whole.
Think about it this way, when you retire, you are going to need money to sustain you for a long time after you stop being able to work, so while you're working, you need to save that money up. That money can just sit in your bank account doing nothing for anyone, or you can invest it in a business that lets them use those resources now and lets you get your retirement money back 30 years from now when you need it (though in reality that's spread across hundreds of companies to reduce risk). That's how investment can be a net benefit to society and make for a better use of resources, characteristics not present with landlords and housing investments.