The EHS defines these as “homes that are primarily used as holiday homes (by family, friends or let to others as a holiday let) or are occupied while working away from home.”

This clearly doesn't include regular rental properties, so I don't see how it shows landlords are not a problem.

Well there are 4 million rental properties in the UK so your numbers don't quite add up. A very significant number of properties are purchased and rented as investment assets. No one is saying that ending landlord property portfolios will magically fix the issue overnight, but it certainly adds inflationary pressure on house prices and prices normal people out from being able to purchase homes. Houses should be homes first, not speculative assets.

Through taxation policy it should be possible for a family to move home and rent their old one, or rent an inherited one whilst waiting to sell, etc without much additional burden. But we shouldn't encourage private entities pricing out normal people from homes and forcing entire generations to only be renters.

The economic damage from high house prices and lack of generational wealth is way too expensive to allow this portion of the market to continue. Simultaneously we need to be building a fuckload more homes, yes, but there being other things to change shouldn't dismiss other helpful ideas.