That feels kind of all-or-nothing. Environmental issues are part of the problem destabilizing societies. Overall, the poisoning of the environment is much worse and much less contained with fossil fuels than with nuclear power. Distant future societies might have no knowledge of nuclear storage sites and a few people might even die before they realize they need to stop breaking into the underground barrels. But a lot more people will die from the environmental havoc that we're causing with fossil fuels. And they can't just stay away from the barrels to avoid that one.
Just to be clear, I think wind and solar (and geothermal where appropriate) are the best ways to get off of fossil fuels. They've gotten a lot cheaper than nuclear so it doesn't make much sense to build new reactors. But it also doesn't make much sense to shut them down if nuclear waste is the only issue.