I don't think most people really think a vacation home or single rental makes you a leech, but when your entire portfolio is single family 2-3 bedroom houses you're kind of sucking the air out of the local market. I know in my city there are tons of "small landlords" with less than 30 houses, half of them don't even live in the city but in ex-urbs an hour away. They don't do anything for the neighborhoods and see their income stream as completely legitimate.