Well, they're absurdly easy to accidentally maim yourself with and unless you're precognitive the blaster deflection feature is virtually useless, so just packing a gun is usually a better move. Kyber mines also didn't exist until the Empire, and they were heavily restricted with only the Emperor himself, Vader, and the Inquisitors getting any at all that weren't being sent to the Death Star. Jedi still go on a bit of a quest to find a crystal in the new continuity. And if you and the crystal don't get along for whatever reason the blade will be heavy and unwieldy. Most of the kyber crystals don't care much, but the older ones have personalities, like the one in the Darksaber.

Wait, why do people actually believe that you need to be Force sensitive to activate a lightsaber?

You're telling me Luke, who would not even begin asking questions about what the Force was for several more scenes, was handed a lightsaber from Ben Kenobi - a device he has never seen in person before, and activated it?

When Han solo cut open the belly of Luke's tauntaun, was he using the Force to activate his lightsaber? Or was a delirious Luke on the brink of death helping him so that they both didn't freeze to death?

The on/off switch is the only thing that actually makes any sense.

My current (possibly wrong) understanding is that jedi keep the planet with the crystals secret, so that not just anyone can go make a saber. On top of that, the Force allows a user to kinda “extend” their perception, which allows for things like reflecting blaster shots, as well as being hyper aware of their personal space during saber duels. So a trained jedi can “see” things like how far a blade is from hitting them, even if they can’t actually see, and that determines how they move.

So it’s more that taking one would be like robbing a special forces soldier, and playing with one would most likely kill or dismember you.