Simply put, the crime from drugs arises only from society criminalizing the supply. People want to have a good time and where there is a demand a supply follows. But the demand doesn't go away because of the criminalization, just as you said. The substitute for illegal drugs (cocaine, heroin, ...) are legal drugs (alcohol, nicotine, ...). But as many users will tell you illegal drugs provide a whole different level of high. The substitute therefore is imperfect and we get to the situation we are at today.

Thinking about guns: Which demand do they satisfy? Security? Every nation on earth operates a police force that is trained in and tasked with upholing law and order (to which extend they do is a different debate). Therefore the average citizen does not need a weapon for self-defense. Skillfull competition? There are many other fields people can compete in. If it has to be loud and make a bang there is i.e. rocket assisted model aviation just to name one. Guns can therefore be reasonably substituted by other means.