But only for the 4 weeks a year you spend in unusually cold weather, the other 48 it's more efficient.

It's not like truly arctic places are a reasonable application but the overwhelming majority of our population lives south of Quebec and north of Wellington. So it's not a relevant point, everyone in the Arctic can just use resistive heating or burn fuel, and if we get everything else on heat pumps we reduce our enegy use by a factor of 2-3 regardles.

Sure, for a few days a year it might get as bad as resistive heating. How horrible! So you don't get 3.2x total throughout the year, you get 3.1x. It's a non-issue.