It's cooler underground because it's well insulated. That also means it's incredibly difficult to get rid of any waste heat. With artificial lighting, electronics, hell even just people moving around, that's a ton of heat being out that needs to go somewhere. Not to mention the amount of ventilation that would be required just to have breathable air. Are you imagining people driving around on underground roads?

Mad skylights, my dude.

But then you wouldn't get the protection from the elements being underground gives you, you would just be more vulnerable to flooding by creating a deep valley. Unless you mean placing glass over the entire thing, but that would be very expensive, it would be very heavy (so needs to have even stronger supports), and would need to be cleaned by somebody on the surface.

It is almost universally known that being underground is cooler.

It depends on where you are. In very sunny places, yes, it would be cooler. However, when you are deep underground, it can get very hot as well! Many construction projects involving digging out deep tunnels and holes have been cancelled because it became too hot for the workers and machines to operate!

why couldn't the structure of the underground dwelling be similarly strengthened [for earthquakes]?

Yes, you would, I mentioned this point needed to be considered. It's a larger consideration when you're underground, as there needs to be additional supports to prevent the chamber and tunnels from collapsing (you might have seen the braces of those mining tunnels, think those but on a much larger scale)

Bro we already built that.

But with the solar panels idea (or your skylights w/ glass), you can't really use the current infrastructure. You would need special roads, homes, etc. that go around them. Also, I would assume that less urbanised areas would need more development to be able to manage an array of solar panels or glass skylights.

Also, I treated this question more like what if humans never built massive cities above ground and we became underground dwellers (perhaps sometime after heavy construction equipment catched on),