At that point I would definitely look at the mileage. You also need to consider the wear/tear on the suspension, transmission, brakes, steering, etc. I've seen far too many people put off regular suspension maintenance leading to other complications later on down the line. If a worn suspension can fuck up motor mounts and tie rod ends, I'd imagine they can rattle those battery packs as well. So I wouldn't think of mileage so much as the "age" of a car so much as it's "experience". Much like how we measure the "age" of aircraft in flight hours. More flight hours = more stress on the frame.

So yeah, more getting to the actual spirit of your inquiry; mileage is gonna matter more to longevity for the vehicle as a whole over charge cycles. On top of that, the battery on the higher mileage vehicle is at higher risk of some form of mechanical failure despite same capacity/lower number of charge cycles.