It makes more sense when the isekai is the character getting transported away and immediately placed in mortal danger over and over like Wizard of Oz. Dorothy has a choice of a provincial life where she might grow up and a world where flying monkeys attack her every chance they get and she's never not on the run the entire story.

If I were isekai'd I would probably be extremely curious about whether I had a connection to my old world. But I wouldn't necessarily leap to go the first chance I got.