Believe me, I've been there. I've thought what you think for decades, but at some point it clicked and I knew.

My point is that there is no part that makes the one that died “the only you”. Your “point of view” is an illusion. Your belief in your ego being a unique continuous thing is product of how our brains functions, not a fact.

Think about the freeze example some more. Think about what would happen to “you” if you ship-of-theseus’d your brain while frozen by dividing all molecules randomly into two piles, adding copies of the respective other pile, and reassembling everything: what molecule holds the “you” particle?