I'm imagining someone switching to Pathfinder 2e, not telling anyone, and whenever it comes up they say it's house rules.

I also feel like D&D is kinda hard to learn and has decades of terminology and baggage that contribute to that. Ah well

This is very often a thing people believe! Especially if the other system they're looking at is like Pathfinder (similarly complex) or some close D&D relatives that have a different set of arbitrary numbers. Like, in this game a 15 strength is +3! We have 50 feats with similar names but different behaviors! They might not even realize that not every game has six stats, or long lists of "feats", or anything even like "feats". And a lot of games (most of them?) don't have weird tables and mappings.

Like if you're playing Fate Core, and you want to burgle, you just your burgle score. One number.

But I think a lot of the time when people present that kind of resistance, it's coming from an emotional place. Telling them facts isn't going to do much. They might feel embarrassed about not being good at the new game. They might feel bad about spending $80 on the D&D books and unusual dice when the new game has a free book and just uses d6. That kind of stuff. Unfortunately, most people aren't really introspective enough to surface those feelings quickly and accurately. (I include myself in "most people" there, sadly.)

I had a guy in an old group that once with full sincerity said "The best thing about D&D is we can just try out different house rules, and if we don't like them we can change something out." Like, my guy, that's not a unique property of D&D. If anything, D&D is harder to homebrew because it has oddly specific rules and assumptions.