I grew up in the north, and never adapted after moving to the south. I would bicycle every day and still look like someone poured a bucket of water on me just being outside for 20 minutes walking. It has taught me to value the feeling of being a little too cold.
This winter I was in the north again, and was reasonably comfortable in -20C, but not nearly as comfortable as the indigenous people who can walk around without gloves and still use a phone without any issue.
You adapt to some extent, but I think everyone is built different at a metabolic level, something to do with the way blood circulates in your capillaries or something.