Actually, solar does kill more than nuclear.

In raw numbers, sure. But that's because solar installations are far, far more common than nuclear installations.

Instead of looking at raw totals, you need to look at deaths/injuries per gigawatt-hour produced. Looking at it that way, I don't think solar would come out as the more dangerous of the two.

(Deer kill more people than bears. But that's only because people meet and interact with deer much more often. I'd rather be locked in a cage with a deer than locked in a cage with a bear.)

Also, if you're going to include mining and refinement in solar panels, you'd better be including mining and refinement for nuclear plants as well. Not just for the nuclear fuel, but also for all the metals, concrete, and other materials that are necessary to build a plant and deal with its eventual waste products.

To be fair, though, that would be extremely difficult to calculate. Suppose a miner working in a copper mine gets run over by a mine truck on the job. Most of the copper from that mine goes toward making copper wires. A tiny portion of those wires were used in the construction of a nuclear power plant. Another tiny portion of those wires were used to connect solar panels. And the vast majority of those wires were used for different purposes entirely, not related to power generation of any kind. Which energy source gets counted for that worker's death?