Let's just ignore the recent and rampant suppression of unionization to avoid improving work conditions then

It's a matter of scale, and what they do with the money. Wealth hoarding has reached the point where it's a cause of inflation. The top 1% (and even moreso, the 1% of that 1%) have hoarded ao much wealth, and do nothing but essentially sit on that wealth, that it makes life more expensive for all of us in the working class.

Strictly speaking, yes they absolutely are hurting someone. Wage theft is a significant issue we face today.