No. How you choose to spend your free time is your thing. Some people decided not to have kids, maybe even for that exact reason. Some people decided to have kids, even though they knew kids would take away their time. It's just how everyone decides to spend their time, and there's nothing wrong with it.