The last time I saw this come up there were approx. 500 calorie difference between the top and the bottom for similar fitness/build of the genetics curve.
500 calories is a pretty big workout, but its at the opposite ends of the curve, most people will be around 250 calories of the middle.
What is far more important is the genetic and learned response to feeling full and hunger. Saying its easy for people to just switch track on that is frankly an awful take, the current weight loss drugs work for a reason, while you carry on taking them. People rebound from often impressive weight loss in huge numbers for a reason, and its not because they forget how to calorie count or exercise.