The key part is that there was a massive amount of plant and animal life, so there was plenty to forage. Like 80% of all life compared to 10k years ago is dead now and we just have scraps left now.

By golly you're right, the consensus is that people in simple foraging societies worked about 6.5 hrs/day. Scholars seem to believe medieval peasants worked more like 8-16 hrs/day, depending on how long daylight lasted - but taking frequent rest breaks, festivals and other holidays.