The confusion here is there are a few different ways of playing D&D and many different types of DMs out there.

This is an important point. There's not really a "right" way to play so much as a "right way for your group".

I don't think D&D specifically does a good job of guiding groups into finding what they'll enjoy. It comes loaded with a lot of assumptions, and then different players can sit down at a table without realizing how different their axioms are.