I noticed something similar with video. Like, if I am paying attention, the difference between the highest quality encoding and the next level is usually visible.

However, I have a harder time telling the difference if I don't do a side by side comparison.

And even when I can easily tell the difference, once I'm watching the thing, I get into the story and I don't care anyways.

Obviously a slightly different criteria compared to music, but people do make a big deal out of stuff that even they don't actually care about.