I imagine whether or not you can taste the rock depends on how well the candy is made. Cheaper rock is coarser and contains more regular rock grit, the more expensive stuff is milled finer and is almost pure magic crystal.
I imagine it would be kind of a touristy thing, like there might be street vendors selling rock candy in a town market, and some fancier confectioners that all claim to have the finest rock candy in the whole region.
The final insult to the rocks is the the candy is mostly sold to travelers because most people agree that it's not that good. kinda like licorice, or salt water taffy. Some people love it, and it's a kind-of interesting feeling to have the crystals dissolve in your mouth, but most people would just as soon pass after they've had it once.