Don't forget, that they produce immediately useable energy. No heat loss, due to steam turbines.
And then there is the timespan that nuclear waste stays harmful. OPs "indestructable" container have to stay indestructable for millions of years.
If we assume 40 years as a generation, that will be 50,000 generations. The whole history of mankind is only 400 generations.
Edit: Added sources for everyone unable to use the Internet for its intended use.
https://en.wikipedia.org/wiki/Radioactive_waste#cite_note-3
Half lifes: https://en.wikipedia.org/wiki/Isotopes_of_plutonium https://en.wikipedia.org/wiki/Isotopes_of_uranium
Estimating a generation of 40 years was generous: https://en.wikipedia.org/wiki/Generation_time
History and pre history: https://en.wikipedia.org/wiki/Recorded_history