The trend when doing web-development since at least 2014 or so is mobile-first responsive design, which is basically as you describe, and it is really not that hard.

You're right - the only reason that any regular website isn't performant is usually because it's chock full of bullshit. Most of the web is unfortunately riddled with a few problems:

• library bloat. The vast majority of developers (even outside of just web) will include a giant library for 1-2 functions instead of just writing it themselves. I don't really blame them, most of the incentive for development these days has little to do with building things well, but instead focuses on building them quickly. With web dev in particular, it's extremely common to import a few thousand npm packages from dubious sources.
• on the web especially, everything seems to be running 3-4 different tracker networks, ad tech bullshit, anti-adblock stuff, click/scroll/jackers, etc. I'd hazard a guess that the vast majority of the regular web, the content is less than half of the actual function of the site.
• It's also extremely common in web dev to use some cool kid toolset designed to recreate photoshop on the web to build a blog or whatever.