You're conflating "not having to work" with "not having to work to hold onto a middle-class standard of living", plain and simple.
My mother-in-law retired with $30,000 in her retirement account, and only gets $1300 a month from Social Security. She has a two-bedroom apartment and she is constantly buying DVDs and crafting stuff she never uses. I won't deny there's probably some money there we don't know about, I honestly don't know how she does it, but compared to that, you're saying $100,000yr is a struggling wage-slave?