The parts of the world with the most population growth are generally also the poorest. Richer countries have fewer children, and within those countries richer people have fewer children.

I'm not saying that your concerns about your quality of life are invalid or that they aren't the reason you personally don't want children, but they don't explain this general phenomenon.